Agents2026~107 minarXiv:2603.29620

Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis

score 8Shuang Chen, Quanxin Shou, Hangting Chen +16Tencent Hunyuan↑ 46★ 57Mar 31, 2026cs.CVhttps://github.com/shawn0728/Unify-Agent

🎯 Pitch

Standard image generators hallucinate catastrophically when asked to draw long-tail cultural concepts like obscure cartoon characters or historical figures—they simply don't know what these things look like. Unify-Agent solves this by acting like a researcher: it first identifies what visual knowledge it's missing, then actively searches for textual and visual evidence online before generating the image. The resulting system more than doubles the base model's factual accuracy on rare concepts, demonstrating that giving models the ability to look things up at inference time is far more effective than relying solely on memorized knowledge.

1. Executive Summary

This paper introduces Unify-Agent, the first end-to-end unified multimodal agent that reformulates text-to-image generation from a passive prompt-to-image mapping into an active, inference-time sequential decision process. Built on the Bagel unified multimodal model and trained on 143K curated agent trajectories, Unify-Agent executes four tightly coupled stages — Think (cognitive gap detection to identify missing visual knowledge), Research (sequential textual-then-visual evidence acquisition from external sources), Recaption (transforming retrieved multimodal evidence into generation-optimized textual specifications by disentangling identity-preserving constraints from scene-compositional constraints), and Generate (evidence-grounded image synthesis) — achieving an overall score of 73.2 on the newly introduced FactIP benchmark (2,462 prompts across 12 long-tail factual categories), surpassing its base model Bagel by over 22 points and outperforming generation-only baselines such as FLUX.1-dev (28.9) and SD-3.5-large (27.5). The approach sets state-of-the-art records among open-source unified models on WiSE (0.77 overall), KiTTEN (4.08 overall), and T2I-FactualBench while demonstrating that visual injection yields substantially larger gains than textual injection alone (16.6 vs. 3.0 absolute improvement on rare IP generation), and that the joint availability of low-level VAE generative latents and high-level ViT semantic tokens creates a mutually reinforcing synergy during evidence recaptioning — establishing that architectural unification of understanding and generation directly improves factual visual grounding for long-tail concepts.

2. Context and Motivation

The Core Problem: Text-to-Image Models Don't Know What Rare Things Look Like

The fundamental problem this paper tackles is straightforward but pervasive: current text-to-image models fail catastrophically when asked to generate images of rare, long-tail, or knowledge-intensive concepts that weren't adequately represented in their training data. This isn't a failure of visual rendering quality — the models can produce photorealistic, aesthetically pleasing images. It's a failure of knowledge: the model simply doesn't know what the target entity should look like, which attributes are identity-defining, or how those attributes should manifest under compositional instructions.

The paper frames this as a closed-book vs. open-book distinction. Standard T2I models operate in a closed-book regime: all their knowledge is frozen in parametric memory at training time. When a user prompts for "Dudoo by TNT SPACE wearing a popcorn bucket costume" or "William Butler Yeats writing by oil lamp in his stone cottage," the model must reconstruct the visual identity from whatever it memorized during pretraining. For popular concepts (e.g., "Mickey Mouse," "Eiffel Tower"), this works reasonably well because the training data contains abundant examples. For the vast universe of long-tail entities — obscure cultural icons, regional mascots, historical figures with limited photographic records, niche intellectual properties, specific product designs — the parametric knowledge is either absent, incomplete, or corrupted by training-data noise.

This gap matters for several practical reasons the paper identifies:

Cultural and factual representation: As image generation moves from artistic novelty to practical tooling (content creation, education, journalism, historical visualization), the inability to faithfully depict real-world entities undermines trust and utility. A history textbook illustration of Max Weber should actually look like Max Weber, not a generic bearded professor.
IP and brand-sensitive generation: The creative industries increasingly use generative AI for concept art, marketing materials, and product visualization. If a model can't faithfully render a specific action figure, mascot, or branded design, it fails at the core task.
Knowledge-intensive domains: Scientific visualization, architectural rendering, and cultural heritage preservation all require models that can accurately ground generation in factual world knowledge — not just produce something that looks plausible.

Why This Is Fundamentally Different From Standard T2I Challenges

The paper draws a sharp distinction between two types of generation failures (Section 2.3):

Render-quality failures are what most T2I benchmarks measure: blurry outputs, incorrect composition, poor aesthetic quality, failure to follow prompt instructions about scene layout or style. These are failures of the generative process itself.

Knowledge-gap failures are fundamentally different: the model produces a visually high-quality image that is factually wrong because it never learned what the target entity looks like. The paper's motivating example (Section 3.2, Figure 2) demonstrates this clearly: when prompted with rare IP concepts, the base Bagel model produces coherent, well-rendered images that simply don't depict the right entity. The generated Scottie Pippen doesn't look like Scottie Pippen; the generated Dudoo figure doesn't have the right facial features; the generated Gregg Popovich scene has the wrong jersey details.

This distinction is crucial because it means the bottleneck isn't generation quality — it's knowledge access. Improving the diffusion process, scaling the VAE, or adding more aesthetic training data won't fix these failures. What's needed is a mechanism to acquire and integrate external world knowledge at inference time, which existing T2I architectures fundamentally lack.

Conflicting Evidence: Why Existing Solutions Are Insufficient

The paper identifies three categories of prior approaches and systematically explains where each falls short.

1. Unified Multimodal Models (UMMs) are still closed-book systems.

Recent architectures like the Janus family, Show-o, Emu3, and Bagel itself represent genuine progress by unifying visual understanding and image generation within a shared backbone (Section 2.1). This architectural unification creates the potential for world knowledge to inform synthesis — the model can reason about visual concepts using its language understanding capabilities. However, the paper makes a critical observation:

"Despite this, current UMMs remain strictly closed-book systems. Because they rely exclusively on static parametric memory internalized during pre-training, they frequently hallucinate or suffer from identity drift when prompted with rare, long-tail, or world-dependent entities."

In other words, UMMs have the architecture to support knowledge-grounded generation but lack the mechanism to acquire external knowledge at inference time. They're powerful reasoning engines with no access to information beyond what they memorized during training. The paper's experiments confirm this: Bagel scores only 50.9 on FactIP (Table 1, Table 5 baseline row), and adding chain-of-thought reasoning (Bagel-CoT) actually reduces performance to 47.0 — evidence that better internal reasoning without external knowledge access doesn't help (and may even hurt by exposing the model's knowledge gaps more explicitly).

2. Existing agentic T2I systems are brittle, multi-stage pipelines.

The paper is careful to distinguish its approach from prior work that loosely connects LLM planners, retrieval tools, and standalone image generators (Section 2.2). Systems like GenArtist, World-to-Image, and T2I-Copilot attempt agentic image generation but suffer from what the paper calls "fragile multi-API stitching":

"These approaches predominantly adopt a fragile multi-API stitching paradigm. They loosely chain together frozen text-only planners, external search tools, and standalone image generators, failing to realize a true end-to-end UMM agentic framework."

This decoupled architecture introduces several specific failure modes:

Cascading errors: If the text-only planner misidentifies the target entity or formulates a poor search query, the entire pipeline fails. There's no feedback loop where the generator can signal that the retrieved evidence is insufficient.
Evidence integration gap: Retrieved text enriches semantic context but rarely specifies the fine-grained visual attributes (exact facial structure, specific costume details, precise coloration) needed for faithful generation. Retrieved images are visually informative but often include irrelevant backgrounds, watermarks, or compositional elements that conflict with the user's specified scene. The paper puts this precisely: "Retrieved text can enrich semantic context, but it rarely specifies the fine-grained visual attributes needed for faithful generation. Reference images, although visually informative, often include irrelevant background elements and may conflict with the user-specified composition."
Modality separation: Reasoning about visual evidence happens in a language-only model that can't actually see images with the fidelity needed to extract identity-defining visual attributes. The generator receives pre-processed references but can't reason about them.

Figure 2 provides quantitative evidence for this integration gap: while visual injection alone improves performance by 16.6 points on rare IP generation, combining text + visual injection yields only 15.4 points — worse than visual alone. This counterintuitive result demonstrates that naive multimodal injection actually interferes with generation quality because raw text introduces redundant or conflicting information that the model can't properly integrate.

3. Knowledge-oriented benchmarks diagnose the problem but don't solve it.

The paper surveys recent evaluation efforts — T2I-FactualBench, KiTTEN, WiSE — that have systematically documented how badly current models fail on knowledge-intensive concepts (Section 2.3). These benchmarks serve an important diagnostic function: T2I-FactualBench shows that even strong models like DALL-E 3 struggle with factual accuracy on single-concept memorization tasks (scoring 55.5 on SKCM), and the gap widens for compositional tasks. KiTTEN reveals that entity alignment (faithfulness to specific visual entities) consistently lags behind text alignment (following scene-level instructions).

However, the paper's critique is that these benchmarks are "fundamentally diagnostic — they quantify the knowledge gap but offer no structural remedy." They measure the problem without proposing how to fix it. The paper positions FactIP (Section 4.2) as filling a complementary role: a benchmark specifically designed to evaluate the type of world-grounded generation that agentic systems should excel at, with particular emphasis on identity consistency and factual faithfulness for long-tail concepts.

How This Paper Positions Itself

The paper's central thesis is that world-grounded image synthesis for rare concepts requires an architectural paradigm shift from closed-book parametric generation to open-book agentic generation, where models actively acquire and reason over external knowledge before synthesizing images. This isn't just "add retrieval to a diffusion model." It's a reframing of what image generation is for knowledge-intensive prompts: an inference-time sequential decision process of identifying knowledge gaps, acquiring evidence, transforming that evidence into generation-compatible specifications, and synthesizing.

The paper positions Unify-Agent as an end-to-end solution that addresses the weaknesses of both prior categories:

Unlike closed-book UMMs, Unify-Agent can access external textual and visual knowledge at inference time, explicitly resolving identity-critical attributes that are missing from parametric memory.
Unlike decoupled agentic pipelines, Unify-Agent integrates reasoning, evidence acquisition, and generation within a single unified architecture. The recaptioning stage — where retrieved evidence is transformed into structured textual specifications — benefits directly from the model's dual VAE+ViT encoding of reference images, enabling it to reason about both high-level semantics (ViT) and low-level visual details (VAE). This synergy is impossible in decoupled systems where the reasoning model has no access to generative priors.
Unlike purely diagnostic benchmarks, the paper builds both the training infrastructure (143K agent trajectories) and the evaluation infrastructure (FactIP) needed to demonstrate that the knowledge gap can be actively bridged rather than merely measured.

The paper also makes a broader architectural claim that extends beyond the immediate application: in a unified understanding-generation model, generation can actually improve understanding. The ablation results in Table 5 show that removing the VAE (which provides low-level perceptual latents) or the ViT (which provides high-level semantic tokens) from the recaptioning stage both degrade performance, with ViT removal causing a larger drop (Overall score from 73.2 to 61.4). This suggests that the benefit of unification isn't just parameter sharing — it's the joint availability of complementary visual representations that together enable more faithful evidence interpretation. This insight has implications for UMM architecture design beyond the specific agentic pipeline.

The Practical and Theoretical Stakes

The paper frames its contribution as addressing a growing tension in the field. On one side, T2I models have achieved remarkable visual quality and stylistic control. On the other side, the expectations for these models are shifting from "generate something that looks good" to "generate something that is factually correct" — a transition driven by real-world deployment in education, journalism, creative tools, and cultural heritage. The paper argues that this transition cannot be achieved by simply scaling pretraining data (there will always be long-tail concepts) or improving generation quality (the bottleneck is knowledge, not rendering). An architectural solution is required.

The theoretical significance lies in the formal reframing of world-grounded generation as inference over an augmented state space (Equation 3, Section 3.3), where the generative process is factorized into gap detection, evidence acquisition, recaptioning, and synthesis. This factorization makes explicit what was previously implicit: that for knowledge-intensive prompts, the model must reason about what it doesn't know before it can generate faithfully. This connects to broader themes in AI — the value of explicit uncertainty modeling, the power of tool use, and the contrast between memorization and reasoning — while grounding them in a concrete, evaluated system.

3. Technical Approach

3.1 Reader Orientation

Unify-Agent is a unified multimodal model fine-tuned to behave as an agent — it can think about what it doesn't know, actively search for information on the web (both text and images), critically evaluate what it finds, distill that evidence into a precise generation specification, and then synthesize an image grounded in that external knowledge. The system solves the problem of world-knowledge gaps in text-to-image generation: when a user asks for a rare or long-tail concept (an obscure mascot, a historical figure with limited photographic record, a specific product design), the model doesn't just guess from its parametric memory — it goes and looks up what the thing actually looks like, then uses that evidence to guide generation. The "shape" of the solution is a sequential pipeline where reasoning, tool use, evidence transformation, and visual synthesis are all handled by the same model weights, trained end-to-end on trajectories that demonstrate the full agentic workflow.

3.2 Big-Picture Architecture (Diagram in Words)

Unify-Agent has five major components, all instantiated within a single Bagel unified multimodal model:

Base Unified Multimodal Model (Bagel) — the pretrained foundation that natively supports both autoregressive text generation and continuous image synthesis via a Mixture-of-Transformers architecture with dedicated understanding and generation experts. It processes visual inputs through a dual encoding: a ViT (Vision Transformer) for high-level semantic tokens and a VAE (Variational Autoencoder) for low-level perceptual latents.
Think Module (Cognitive Gap Detection) — Given a user prompt, the model explicitly reasons about which identity-critical visual attributes are missing from its parametric memory, producing a structured assessment of knowledge gaps that determines whether external evidence acquisition is needed.
Research Module (Sequential Multimodal Evidence Acquisition) — The model formulates search queries, calls external tools (text search and image search), and critically evaluates retrieved results. Textual evidence is acquired first to establish semantic grounding; visual evidence follows, with retrieved images scored along four quality dimensions and only the top two retained as visual anchors.
Recaption Module (Evidence-Grounded Specification Generation) — Rather than passing raw retrieved text and images directly to the generator, the model transforms the multimodal evidence into a structured textual recaption that explicitly disentangles identity-preserving constraints (facial features, defining visual traits) from scene-compositional constraints (pose, environment, style). This recaption becomes the executable specification for synthesis.
Generate Module (Evidence-Grounded Image Synthesis) — The final image is synthesized conditioned on the recaption and the selected visual anchor images, using the model's native flow-matching generation pathway. Critically, the noisy reasoning history and raw textual evidence are masked out during this stage so that only the refined specification drives synthesis.

Information flows sequentially: prompt → Think (gap detection) → Research (text search, then image search with quality filtering) → Recaption (constraint extraction and structured rewriting) → Generate (flow-matching synthesis conditioned on recaption and visual anchors). The entire trajectory — including search queries, retrieved evidence, reasoning traces, and the final recaption — is supervised during training so the model learns not just what to generate, but how to acquire and reason about the knowledge needed to generate faithfully.

3.3 Roadmap for the Deep Dive

First, the base architecture (Bagel and its dual-loss training): Because Unify-Agent is built on Bagel, understanding the base model's text generation objective (next-token prediction) and image generation objective (flow matching in VAE latent space), plus the critical dual VAE+ViT visual encoding design, is essential for understanding why the recaption stage works and why the agent can reason about visual evidence with high fidelity.
Second, the motivating evidence and problem formulation: The paper's preliminary study (Figure 2) showing that visual injection helps but naive multimodal injection is suboptimal — and the formal factorization of world-grounded generation into an augmented state space (Equation 3) — provide the theoretical and empirical justification for the agentic pipeline design.
Third, the training data construction pipeline: The 143K agent trajectories don't exist naturally; the paper builds them through a three-stage process (IP collection and prompt generation, multimodal research trace construction with a teacher agent, and evidence-grounded recaption annotation with generation-based verification). Understanding how these trajectories are synthesized is crucial because they define what the model learns to do.
Fourth, the supervised fine-tuning procedure: How the trajectory data is packed into interleaved sequences, how the dual text+image loss is applied with specialized attention masking, and what hyperparameters and training configurations are used.
Fifth, the inference-time agentic pipeline: The four-stage Think → Research → Recaption → Generate process, with detailed mechanics for each stage including the sequential text-then-visual search ordering, the four-dimensional image quality filtering, and the constraint-disentangling recaption design.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and methods paper whose core idea is that world-grounded image synthesis for long-tail concepts is fundamentally a knowledge access problem, not a rendering quality problem, and that an end-to-end unified agent — trained on trajectories that demonstrate the full think-search-recaption-generate workflow — can actively bridge the parametric knowledge gap at inference time by acquiring external evidence and transforming it into generation-optimized specifications.

The Base Architecture: Bagel's Mixture-of-Transformers with Dual Visual Encoding

Unify-Agent is built on Bagel, a unified multimodal model that jointly handles visual understanding (producing text responses conditioned on images and text) and visual generation (producing images conditioned on text and optionally reference images) within a single shared architecture. Understanding Bagel's design is essential because several of Unify-Agent's key capabilities — particularly the ability to reason about retrieved reference images with high fidelity during recaptioning — depend directly on architectural properties of the base model.

Mixture-of-Transformers (MoT) backbone. At its core, Bagel uses a Mixture-of-Transformers architecture that disentangles understanding and generation through dedicated experts (Section 3.1). Rather than having a single set of parameters that handles both tasks, the model routes tokens through different expert modules depending on whether the current task involves understanding or generation. This design choice is motivated by the observation that understanding and generation place different demands on the model: understanding requires reasoning about semantic content, relationships, and factual consistency, while generation requires precise control over continuous visual attributes like color, texture, and spatial layout. The MoT architecture allows the model to develop specialized capabilities for each without destructive interference.

Dual visual encoding: ViT + VAE. Visual inputs entering Bagel are encoded through two parallel pathways — this dual design is one of the paper's most architecturally significant choices and directly enables the recaption-to-generation synergy that distinguishes Unify-Agent from decoupled pipelines.

The Vision Transformer (ViT) — specifically SigLIP-SO400M-14 with NaViT — processes images into high-level semantic tokens. The ViT captures global context, object identity, attribute relationships, and scene composition. It operates on image patches (patch size 14) and produces tokens that are compatible with the language model's autoregressive processing, making them suitable for multimodal reasoning tasks where the model needs to understand what is in an image, identify entities, and reason about their properties. During training, ViT inputs are transformed to a resolution of (378, 980) as specified in Table 6.

The Variational Autoencoder (VAE) — specifically the FLUX VAE, kept frozen during Unify-Agent training — encodes images into low-level continuous latents. Unlike the ViT's semantic tokens, the VAE latents preserve fine-grained perceptual details: color distributions, texture patterns, material properties, local geometric structure. These latents are what the image generation pathway operates on (via flow matching), but critically, they are also available during the understanding/recaptioning stages. The VAE operates with a latent patch size of 2 and inputs are transformed to (512, 1024) resolution (Table 6).

The paper explicitly argues that having both representations available simultaneously is what enables high-fidelity evidence interpretation during recaptioning (Section 6.4): the ViT provides the semantic understanding needed to identify who or what is in a reference image, while the VAE provides the perceptual detail needed to describe exactly what they look like. In a decoupled system where retrieval is handled by a text-only LLM calling an API, this dual representation is impossible — the LLM can only "see" what was retrieved, not reason about low-level visual attributes.

Text generation pathway (understanding expert). The understanding pathway is formulated as standard autoregressive next-token prediction. Given a multimodal input context $C$ — which can include interleaved text and ViT-encoded image tokens — the model generates text token by token, with each token's probability conditioned on all preceding tokens and the full multimodal context. The training objective minimizes the negative log-likelihood:

$\mathcal{L}_{\text{text}} = -\sum_{t=1}^{T} \log p_\theta(x_t \mid x_{<t}, C)$

where $x_t$ is the target text token at position $t$ , $x_{<t}$ denotes all tokens preceding position $t$ , $C$ is the full multimodal input context (text + ViT image tokens), and $\theta$ represents the model parameters.

What it computes: The sum of per-token surprisal (negative log probability) across the entire text sequence. At each position, the model looks at everything that came before — both text and images — and predicts a probability distribution over the vocabulary. The loss penalizes the model based on how low a probability it assigned to the actual next token. Lower loss means the model is better at predicting what text should come next given the multimodal context.

Why this form: Negative log-likelihood is the standard maximum-likelihood objective for autoregressive sequence models. It encourages the model to assign high probability to the observed text sequence while (implicitly) pushing probability mass away from other possible sequences. This is appropriate because the training data consists of supervised trajectories where we want the model to exactly reproduce the demonstrated reasoning, search queries, and recaptions. Alternative objectives like contrastive losses would teach the model to distinguish good from bad outputs but wouldn't provide the fine-grained token-level supervision needed to learn structured formats (XML tags, tool call syntax, recaption structure).

Image generation pathway (generation expert). The generation pathway synthesizes images through a rectified flow process operating in the continuous latent space of the frozen VAE. Rather than generating pixels directly, the model works with compressed latent representations where the VAE encoder has already captured the essential visual structure. The generation is conditioned on the same multimodal context $C$ that feeds the understanding pathway, meaning the model can generate images guided by both text instructions and reference images.

The training objective for image generation is a latent flow-matching loss:

$\mathcal{L}_{\text{image}} = \mathbb{E}_{t \sim \mathcal{U}(0,1),\, z_t} \left\| u_\theta(z_t, t; C) - u^*(z_t, t) \right\|_2^2$

where $t \sim \mathcal{U}(0,1)$ is a continuous timestep sampled uniformly from the unit interval, $z_t$ is the latent state at time $t$ (produced by mixing clean latent with noise according to the flow schedule), $u_\theta(z_t, t; C)$ is the velocity field predicted by the generation expert given the current noisy latent and the conditioning context, $u^*(z_t, t)$ is the target velocity field (the ground-truth direction from the noisy latent toward increasing probability under the data distribution), and $\|\cdot\|_2^2$ is the squared Euclidean norm.

What it computes: At training time, a clean image is encoded to a latent $z_0$ via the VAE encoder. A timestep $t$ is randomly sampled, and the latent is partially noised to produce $z_t$ (closer to pure noise when $t$ is near 0, closer to the clean image when $t$ is near 1). The model is given $z_t$ , $t$ , and the conditioning $C$ , and must predict the velocity $u_\theta$ — the direction and magnitude of change needed to move $z_t$ toward the clean latent. The loss is simply the mean squared error between the predicted velocity and the true velocity $u^*$ (which is known from the flow specification). At inference time, generation starts from random noise and iteratively follows the predicted velocity field to produce a clean latent, which is then decoded to pixels via the VAE decoder.

Why this form: Flow matching provides a simpler and often more stable alternative to diffusion models. The key advantage over score-based diffusion is that the target $u^*$ is a straight-line velocity field — it directly points from noise to data — rather than the score function (gradient of log-density) which can be more complex to estimate. The MSE loss is appropriate because the target is a continuous vector; the squared error penalizes large deviations quadratically, which encourages the model to learn a smooth velocity field. This is important for generation quality because small errors in early timesteps can compound into large visual artifacts.

Critical design choice: VAE is frozen. Table 6 specifies that the FLUX VAE weights are "Frozen" during Unify-Agent training. This means only the ViT encoder, the language model backbone, the multimodal connectors, and the generation expert are updated; the VAE encoder and decoder remain exactly as they were in the pretrained Bagel checkpoint. The rationale is likely twofold: (1) the VAE already provides high-quality latent representations and further training on a relatively small dataset (143K examples) could destabilize its encoding, and (2) freezing the VAE ensures that the latent space is stable, which is important for the flow matching objective since the velocity field is defined relative to a fixed latent geometry.

How the dual objectives are combined during Unify-Agent training. In standard Bagel training, text-only samples contribute only to $\mathcal{L}_{\text{text}}$ and image-generation samples contribute to $\mathcal{L}_{\text{image}}$ (with accompanying text tokens also contributing to $\mathcal{L}_{\text{text}}$ ). For Unify-Agent, the paper extends this to handle interleaved agent trajectories where a single training sample contains text reasoning, tool calls, recaption output, and an image generation target. The combined loss is simply the sum:

$\mathcal{L}_{\text{SFT}} = \mathcal{L}_{\text{text}} + \mathcal{L}_{\text{image}}$

with both components equally weighted at 1.0 (Table 6). Supervision masks determine which positions in the packed sequence contribute to which loss: text-only segments (thinking, research, recaptioning) contribute only to the cross-entropy text loss, while image-generation segments contribute to both the cross-entropy loss (for the text tokens that describe or precede the image) and the flow-matching MSE loss (for the image latent itself).

Special token reweighting. Table 6 notes that "CE Loss Special Token Reweighting" is enabled with a weight of 3.0 on designated special tokens. This means that tokens like <thinking>, <tool_call>, <recaption>, and other structural markers in the agent trajectories receive 3× higher loss weight than regular text tokens. The motivation is to ensure the model reliably learns to produce correctly structured output formats — a misshapen tool call or a recaption outside its XML tags would break the agentic pipeline at inference time, so the model is penalized more heavily for errors on these format-critical tokens.

Motivating Evidence and Problem Formulation

Before presenting the full agentic pipeline, the paper establishes both empirical and theoretical foundations for its design choices.

The motivating study: knowledge injection helps, but naively. Section 3.2 presents a "preliminary training-free study on 200 examples sampled from three major categories in FactIP Benchmark: scene, character, and object." Using the base Bagel model without any fine-tuning, the authors compare four inference settings:

Prompt-only: Bagel generates directly from the user prompt with no external knowledge.
Text injection: IP-related background information is appended to the original prompt before generation.
Visual injection: Two ground-truth reference images of the target IP are provided as conditioning to Bagel.
Text + visual injection: Both text and visual evidence are provided simultaneously.

The results (Figure 2) reveal three key patterns:

Both text and visual injection improve over prompt-only. Text alone yields an absolute improvement of 3.0 points on the overall metric, while visual alone yields 16.6 points — a dramatic difference that establishes visual evidence as the primary driver of factual improvement.
Visual injection dominates text injection across all categories. For characters, visual injection improves by 16.3 points vs. text's 3.1; for objects, 16.4 vs. 2.6; for scenes, 15.0 vs. 4.0. This is intuitive: a reference image directly shows what the entity looks like, while text descriptions often fail to capture the specific visual attributes that define identity.
Combining text + visual yields LESS improvement than visual alone. The joint injection achieves only 15.4 points overall improvement compared to visual-only's 16.6. The paper interprets this as evidence that "raw text injection may introduce redundant or weakly visual information, which can interfere with instruction following under complex prompts; conversely, raw visual injection, while effective at anchoring identity, may over-constrain the generation process and reduce flexibility in attribute manipulation or compositional reasoning."

This last finding is the critical empirical motivation for the recaption stage. If naive multimodal injection is suboptimal, then the model needs an intermediate processing step that transforms raw evidence into a generation-compatible form. The recaption is that step.

The formal problem formulation: world-grounded generation as inference over an augmented state space. Section 3.3 presents a probabilistic reframing of what world-grounded image synthesis entails. Standard T2I generation models the conditional distribution $p_\theta(y \mid x)$ — the probability of generating image $y$ given prompt $x$ , relying strictly on the parametric memory $\theta$ that was learned during pretraining. The paper argues that for long-tail concepts, this distribution is essentially degenerate: $p_\theta(y \mid x)$ cannot place high probability on the correct image because $\theta$ lacks the necessary information about what the target looks like.

The augmented formulation introduces four intermediate variables that explicitly represent the knowledge gap and the evidence acquired to fill it. The full joint distribution is factorized as:

$p_\theta(y, c, \tau_t, \tau_v, g \mid x) = p_\theta(g \mid x) \cdot p_\theta(\tau_t, \tau_v \mid x, g) \cdot p_\theta(c \mid x, g, \tau_t, \tau_v) \cdot p_\theta(y \mid c, \tau_v)$

where $x$ is the user prompt, $g$ is the cognitive gap assessment (a latent variable representing what knowledge is missing from parametric memory), $\tau_t$ is the textual evidence trace (retrieved text knowledge), $\tau_v$ is the visual evidence trace (retrieved reference images), $c$ is the evidence-grounded recaption (a structured textual specification for generation), $y$ is the generated image, and $\theta$ represents all model parameters.

What this computes: The joint probability decomposes world-grounded generation into four sequential sub-problems, each conditional on the results of previous stages:

Gap Detection ( $p_\theta(g \mid x)$ ): Given the prompt alone, the model assesses whether parametric memory contains sufficient knowledge to generate faithfully. If key visual attributes are missing, the gap variable $g$ encodes what specifically needs to be looked up.
Evidence Acquisition ( $p_\theta(\tau_t, \tau_v \mid x, g)$ ): Conditioned on knowing what's missing, the model formulates text queries to retrieve $\tau_t$ (semantic background, disambiguation, attributes) and then visual queries (refined by the text evidence) to retrieve $\tau_v$ (reference images).
Evidence-Grounded Recaptioning ( $p_\theta(c \mid x, g, \tau_t, \tau_v)$ ): The model consolidates all available information — the original prompt, the assessed gaps, the text evidence, the visual evidence — into a single structured recaption $c$ that explicitly specifies what should appear in the image and how identity-critical attributes should be preserved.
Visual Synthesis ( $p_\theta(y \mid c, \tau_v)$ ): The final image is generated conditioned on the recaption and the visual anchors, but critically NOT on the full reasoning history ( $x$ , $g$ , $\tau_t$ ). This conditional independence assumption is deliberate: it prevents noisy or redundant textual evidence from interfering with the continuous generative process, which should be guided only by the refined specification.

Why this factorization: The key insight is that $p_\theta(y \mid x, \tau_t, \tau_v)$ — direct generation from prompt plus raw evidence — is known to be suboptimal (as shown in Figure 2). The factorization interposes two crucial transformations: gap detection (which makes explicit what knowledge is needed) and recaptioning (which transforms raw evidence into generation-optimized form). By making these steps explicit parts of the generative process rather than ad-hoc preprocessing, the model can be trained end-to-end to optimize the entire pipeline.

The factorization also enforces a clean information bottleneck at the recaption stage. The final synthesis $p_\theta(y \mid c, \tau_v)$ depends only on the recaption and visual anchors, not on the full chain of reasoning. This has the practical benefit of preventing the image generation from being polluted by the noisy, redundant, or conflicting information that might exist in the raw text evidence or the reasoning traces.

Training Data Construction: 143K Agent Trajectories

The paper does not use existing datasets for training Unify-Agent — it constructs a dedicated corpus of 143K high-quality agent trajectories specifically designed to teach the model the think-search-recaption-generate workflow. This data construction is itself a significant methodological contribution and proceeds in three stages (Section 4.1).

Stage 1: Task Source Curation and Prompt Collection. The first stage builds a large-scale pool of knowledge-intensive intellectual properties (IPs) that are specifically chosen to expose world-knowledge deficiencies in image generation models. The paper emphasizes that these are not popular, frequently recurring entities — they are "relatively long-tail and less frequently represented IPs from the web" collected by "a large group of annotators."

The collected IPs are organized into 12 domains: Celebrity, Animation, Game, Comic, Mythology, Mascot, Animal, Food, Art, Toy, Landmark, and Festival. For each concept, the pipeline retrieves information from BabelNet and Wikipedia, and annotators search for two representative seed images to serve as ground-truth visual references. GPT-4o is used to summarize each IP into structured metadata including identity-defining attributes and factual descriptions. A manual verification step removes samples with incorrect metadata, low-quality seed images (severe watermarks, blurry content), or concepts that are overly common and don't sufficiently challenge factual grounding. After filtering, the pool contains 456K examples — substantially larger than what ends up in the final training set, since many will be discarded during the recaption verification stage.

From this IP pool, the authors construct user prompts designed to test whether a model can preserve grounded identity while performing creative generation. Importantly, the prompts are not simply "generate X" — they include compositional requirements like scene composition, clothing or object specifications, atmosphere, and photographic or cinematic style. Gemini 3 Pro is used to generate diverse prompts tailored to each category's domain characteristics, with difficulty ranging from direct rendering to challenging cases requiring attribute preservation under significant compositional variation.

The resulting training representation for each sample is:

$\mathcal{D}_{\text{SFT}} = \{(x, \tau_t, \tau_v, c)\}$

where $x$ is the original user prompt, $\tau_t$ is the textual research trace, $\tau_v$ is the visual research trace, and $c$ is the final evidence-grounded recaption. Note that $\tau_t$ and $\tau_v$ supervise evidence acquisition behavior, while $c$ supervises evidence transformation — the model learns both how to search AND how to use what it finds.

Stage 2: Multimodal Research Trace Construction. For each prompt $x$ , the authors generate a supervised trajectory that externalizes the evidence-seeking process into a structured, serializable format. Crucially, this trajectory generation is performed by a strong teacher agent — Claude Opus 4.6 — that actually executes the research process using real external tools. The choice of Claude is motivated by its "strong capability on long-horizon agentic tasks, tool use, and complex research-style workflows."

The trajectory construction follows a deliberate text-first, visual-second ordering. Textual evidence is acquired before visual evidence because textual context resolves semantic ambiguity and establishes high-level grounding, which in turn enables more precise visual search queries.

Textual research trace construction:

The teacher agent first formulates a textual query $q_t$ :

$q_t \sim p_\theta(q_t \mid x, g)$

that is issued to an external text search system, yielding textual evidence $E_t$ :

$E_t = \text{Retrieve}_{\text{text}}(q_t)$

The role of $E_t$ is to provide a compact semantic scaffold — identity disambiguation, categorical background, role-specific context, and other descriptors useful for constraining subsequent visual search. Both the query $q_t$ and the resulting evidence summary $E_t$ are recorded as part of the supervised trace. This means the model is trained not just on what knowledge to use, but on how to formulate effective queries to acquire it.

Visual research trace construction:

Conditioned on the prompt, the gap assessment, and the textual evidence, the teacher agent generates a visual query $q_v$ :

$q_v \sim p_\theta(q_v \mid x, g, \tau_t)$

This query is designed to retrieve images that are both identity-preserving and context-compatible — a substantial improvement over naive name-based retrieval because it incorporates prompt-specific constraints and the semantic grounding from $\tau_t$ .

The initial candidate set is retrieved via image search:

$\tilde{E}_v = \text{Retrieve}_{\text{image}}(q_v) = \{v_1, \ldots, v_n\}$

Now comes a critical quality-filtering step. Rather than using all retrieved images, the paper employs Gemini 3 Flash as a lightweight visual evaluator to score each candidate along four dimensions:

Identity consistency: Whether the image faithfully matches the target IP/concept.
Subject salience: Whether the IP is the dominant, clearly visible subject.
Image clarity: Whether the image is sharp and free of compression artifacts.
Watermark cleanliness: Whether the image avoids heavy watermarks or text-dominant compositions.

Each dimension is scored independently, and the overall score for a candidate image is:

$s(v_i) = \sum_{k=1}^{4} \lambda_k \cdot s_k(v_i \mid x, E_t)$

where $s_k$ is the score for dimension $k$ and $\lambda_k$ are dimension weights (not explicitly specified in the paper but implied to be equal or tuned for quality).

The candidate pool is ranked by this score, and the top two images are selected:

$E_v = \arg\!\text{top}_2^{v_i \in \tilde{E}_v} s(v_i)$

These two images serve as the primary visual anchors for downstream recaptioning and synthesis. Selecting exactly two (rather than one or many) is a deliberate balance: one image might not capture all identity-relevant attributes, while many images could introduce conflicting visual information.

The full multimodal research trace is represented as:

$(\tau_t, \tau_v) = (q_t, E_t, q_v, E_v)$

where every element — both queries and their results — is included in the supervision. This makes the evidence acquisition process fully explicit in the training data, aligned with the factorized term $p_\theta(\tau_t, \tau_v \mid x, g)$ in Equation 3.

Stage 3: Evidence-Grounded Recaption Annotation with Generation-Based Verification. This stage addresses the empirical finding from Section 3.2 that raw evidence injection is suboptimal. Instead of directly passing retrieved text and images to a generator, the pipeline constructs a structured recaption that consolidates the original instruction with the retrieved evidence into a generation-optimized description.

The recaption is defined as:

$c = \mathcal{C}(x, E_t, E_v)$

where $\mathcal{C}$ denotes the recaptioning function. The resulting recaption integrates three complementary sources of information:

Prompt-level compositional constraints: Scene, pose, clothing, atmosphere, rendering style — everything the user explicitly requested.
Semantic context from textual evidence: Identity disambiguation, background consistency, and factual grounding extracted from $E_t$ .
Identity-preserving visual cues from reference images: Specific visual attributes (facial features, costume details, color specifications) grounded in $E_v$ .

The paper emphasizes that this recaption is "not a generic image caption and not a simple paraphrase of the prompt." It is a structured textual specification explicitly designed to control downstream image synthesis. This design is motivated by recent work on prompt rewriting and model-facing specifications like DALL-E 3's caption upsampling, PromptEnhancer, and HunyuanImage 3.0.

Generation-based verification via reject sampling:

To ensure that the annotated recaptions are not merely semantically plausible but actually effective for image synthesis, the pipeline includes a validation step. After producing the recaption $c$ , it is fed together with the two selected reference images into Nano Banana Pro to synthesize a validation image:

$\hat{y} \sim p_\phi(y \mid c, E_v)$

where $p_\phi$ is the image generator used for data construction purposes (distinct from the Unify-Agent model being trained).

The generated image $\hat{y}$ is compared against the ground-truth image of the corresponding IP using GPT-4o as a multimodal judge. GPT-4o evaluates whether $\hat{y}$ is sufficiently faithful to the intended identity and visual concept. If the generated image fails this identity-consistency check, the recaption-to-generation process is repeated, up to five trials total. If no satisfactory result is obtained after five attempts, the entire trajectory is discarded — the failure is treated as evidence that the underlying retrieval or recaption is unreliable.

This reject-sampling strategy serves two purposes. First, it filters out noisy supervision signals that would teach the model incorrect grounding behavior (e.g., recaptions that sound plausible but produce wrong images). Second, it ensures that the final training set contains only trajectories where the recaption-to-generation pathway demonstrably works — giving the model clean, reliable supervision.

After this three-stage pipeline, the authors obtain 143K high-quality trajectory-image pairs from the initial 456K IP pool. The substantial reduction (from 456K to 143K) reflects the stringency of the quality filtering: concepts with unreliable metadata, poor reference images, or recaptions that fail generation verification are all excluded.

Supervised Fine-Tuning on Agent Trajectories

With the trajectory data constructed, the paper fine-tunes the Bagel base model to internalize the full agentic workflow. This is not a simple prompt-tuning or few-shot adaptation — it is full supervised fine-tuning on interleaved multimodal sequences using the dual text+image objectives described in Section 3.1, with several important training design choices specific to the agentic setting.

Sequence packing for heterogeneous samples. Agent trajectories are inherently variable in length and composition: some samples involve extensive text search and reasoning, others may have shorter research phases, and the presence of retrieved reference images varies. To handle this efficiently, the paper uses sequence packing — concatenating multiple heterogeneous training samples into a single packed sequence that fills up to the maximum token limit. This allows the model to process text-only and image-containing samples within a single forward pass, jointly optimizing both objectives.

The relevant hyperparameters from Table 6:

Max tokens per sample: 40,240
Expected tokens per batch: 40,240
Max packed tokens (hard limit): 41,520

These are unusually long sequence lengths — typical language model fine-tuning uses 2K–8K tokens. The 40K token budget reflects the length of full agent trajectories, which include multiple rounds of tool calls, search results, image tokens (which consume substantial context), recaptions, and image generation.

Specialized attention masking. The paper introduces a hybrid attention masking strategy (detailed in Appendix B, visualized in Figure 6) that selectively controls information flow within the packed sequences. This is one of the most architecturally novel aspects of the training design and directly addresses a key challenge: in agent trajectories, not all information should be visible at all stages.

The masking strategy has three regimes:

Textual reasoning and dialog (causal attention): For the text components of the sequence — agentic dialogs, reasoning steps, tool invocations — standard causal masking is applied. Each token can attend to itself and all preceding tokens (both text and image), preserving the temporal and logical flow of the agent's decision process. This ensures the model learns to generate the next reasoning step conditioned on the full history.
Retrieved reference images (full attention): Reference images are represented by a composite of VAE tokens and ViT tokens. Within this composite, full (bidirectional) attention is applied — the VAE and ViT tokens can attend to each other globally. This allows holistic visual feature extraction where low-level perceptual details and high-level semantic features can interact without positional restrictions.
Image generation phase (restricted attention): During generation, the latent flow-matching tokens (VAE noisy tokens for the output image) are specifically restricted to attend ONLY to:
- The preceding recaption tokens (the refined textual specification)
- The retrieved reference image tokens (the visual anchors)
Critically, they CANNOT attend to historical reasoning traces, raw textual evidence, or earlier dialog. This enforced information bottleneck implements the conditional independence assumption from Equation 3: $p_\theta(y \mid c, \tau_v)$ — generation depends on the recaption and visual anchors, not on the noisy upstream reasoning. The paper states this prevents "large segments of high-noise text from interfering with the synthesis process."

This three-regime masking design is what allows a single unified model to handle the full agentic pipeline without the text reasoning degrading the image quality or the image generation being distracted by irrelevant retrieval noise.

Loss application with supervision masks. The combined loss $\mathcal{L}_{\text{SFT}} = \mathcal{L}_{\text{text}} + \mathcal{L}_{\text{image}}$ is applied selectively within each packed sequence using supervision masks:

Text-only segments (thinking, research, recaptioning) contribute only to $\mathcal{L}_{\text{text}}$ .
Image-generation segments contribute to both $\mathcal{L}_{\text{text}}$ (for accompanying text tokens) and $\mathcal{L}_{\text{image}}$ (for the image latent tokens).

The cross-entropy loss includes special token reweighting (weight 3.0 on tokens like <thinking>, <tool_call>, <recaption>) to emphasize correct formatting of structured outputs. This is critical for agentic behavior: a tool call with malformed JSON will fail at inference time regardless of the model's reasoning quality.

Training configuration. The full hyperparameter specification from Table 6:

Hardware: 64 NVIDIA H20 GPUs
Training duration: approximately 10 days, 10,000 gradient steps
Optimizer: AdamW with $\beta_1 = 0.9$ , $\beta_2 = 0.95$ , $\epsilon = 10^{-15}$
Learning rate: $5 \times 10^{-5}$ , constant schedule after 500 linear warmup steps
Gradient clipping: maximum L2 norm of 5.0
Parallelism: FSDP with HYBRID_SHARD strategy
Frozen components: FLUX VAE (both encoder and decoder)
Trainable components: Language model backbone, ViT encoder, multimodal connectors, generation expert

The constant learning rate schedule (rather than cosine decay) and relatively high peak learning rate ( $5 \times 10^{-5}$ ) are notable. They suggest that the fine-tuning is substantial enough to significantly reshape the model's behavior (learning entirely new capabilities like tool use and recaptioning from scratch) rather than just doing a light adaptation.

The Inference-Time Agentic Pipeline: Think, Research, Recaption, Generate

At inference time, Unify-Agent executes the trained agentic workflow as a sequential decision process. The paper describes this in Section 5.2 and illustrates it in Figure 4. The four stages directly instantiate the factors in Equation 3.

Stage 1: Think — Prompt Understanding and Cognitive Gap Detection.

The model begins by analyzing the user prompt $x$ not as a complete specification, but as a partially specified generation request. It performs structured prompt understanding, decomposing the instruction into semantically meaningful factors:

Target identity (who or what is being depicted)
Scene configuration (setting, environment, context)
Stylistic intent (photographic, illustrative, painterly, etc.)
Specific requirements (pose, clothing, props, atmosphere)

Based on this structured decomposition, the model assesses whether faithful synthesis depends on knowledge that is absent, ambiguous, or unreliable in its parametric memory. This process is captured by the latent gap variable:

$g \sim p_\theta(g \mid x)$

which governs whether external knowledge acquisition should be triggered.

To make the gap concrete, the model represents it as a set of missing knowledge units:

$\mathcal{M}(x) = \{m_1, \ldots, m_K\}$

where each $m_k$ is an identity-critical or visually consequential attribute not sufficiently specified by the prompt — for example, facial structure of a historical figure, hairstyle, signature appearance features, object-specific structural details, or scene-specific factual elements.

The crucial design choice is that this stage doesn't merely recognize that an entity is rare or knowledge-intensive. It aims to identify which specific attributes must be recovered to support faithful generation. When $\mathcal{M}(x) \neq \emptyset$ , the model enters the research stage; otherwise, it may proceed directly to generation. This conditional branching (roughly: "do I know enough or do I need to search?") is what makes the system agentic rather than blindly retrieving for every prompt.

Stage 2: Research — Sequential Multimodal Evidence Acquisition.

Conditioned on the detected knowledge gaps, the agent performs external evidence acquisition following a deliberate text-then-visual ordering. The paper explicitly motivates this ordering: "Textual evidence first resolves semantic ambiguity and establishes high-level grounding, which in turn enables more precise and context-aware visual search."

Textual evidence acquisition:

The agent formulates a textual query based on the prompt and the identified knowledge gaps. This query is issued to external search systems (the paper mentions Wikipedia, BabelNet, and web search in the data construction but doesn't specify the exact search backend for inference). The result is a textual evidence trace $\tau_t$ that provides:

Identity disambiguation (resolving which specific entity is meant)
Categorical background (domain-specific context)
Key attributes described in text form
Any other semantic context useful for guiding visual search

The textual evidence is not intended to directly specify visual attributes for generation — it provides the semantic scaffold that makes subsequent visual search more precise.

Visual evidence acquisition:

The agent formulates a visual query that incorporates both the original prompt constraints and the textual evidence. This is a key improvement over naive name-based retrieval: rather than searching for "Dudoo TNT SPACE," the query can incorporate category context ("TNT SPACE Dudoo blind box figure popcorn bucket variant"), making retrieval more likely to return identity-relevant and context-compatible images.

The retrieved candidate images undergo quality evaluation using the same four-dimensional scoring protocol as in training data construction (identity consistency, subject salience, image clarity, watermark cleanliness). The top two images are selected as the visual evidence trace $\tau_v$ .

The paper doesn't specify whether the inference-time visual evaluation uses Gemini 3 Flash (as in training data construction) or the model's own capabilities. Given that Unify-Agent is trained on trajectories that include image quality judgments, it's reasonable to infer that the model performs this evaluation itself as part of the learned agentic behavior.

Stage 3: Recaption — Multimodal Grounding into Executable Specifications.

This is the central design innovation that distinguishes Unify-Agent from both closed-book generation and naive retrieval-augmented generation. Rather than passing raw evidence to the generator, the model transforms the collected multimodal evidence into two types of grounded constraints:

Identity-preserving constraints ( $\mathbf{h}_{\text{id}}$ ): These capture the visual attributes that must remain faithful to the target identity — facial structure, appearance cues, object-defining characteristics, color specifications — as grounded in the reference images. The paper gives the example of "facial traits, hairstyle, signature details" for a character.
Scene-compositional constraints ( $\mathbf{h}_{\text{scene}}$ ): These encode the prompt-specified factors of generation — pose, environment, garment, mood, composition, stylistic presentation — that are independent of identity fidelity but crucial for prompt alignment.

The agent then integrates these constraints into an evidence-grounded recaption:

$c \sim p_\theta(c \mid x, g, \tau_t, \tau_v)$

The recaption serves as an abstraction layer that filters noisy raw evidence and reorganizes it into a generation-compatible representation. The paper provides a concrete example in Figure 4: for a Dudoo popcorn bucket figure, the recaption includes "Preserve the exact facial features of Dudoo by TNT SPACE as shown in image_1 and image_2: a round and plump face shape, fluffy and soft cream-colored/golden creamy hair, a pair of watery big eyes with star-shaped highlights, a healing smile showing neat little teeth, and a fair and delicate skin tone. The character is wearing red and white striped popcorn bucket jacket as shown in image_2, featuring a cute letter logo and star decorations on the chest..."

This example illustrates the critical structure: identity attributes (facial features, hair, eyes) are explicitly grounded in specific reference images ("as shown in image_1 and image_2"), while compositional elements (the popcorn bucket costume, the scene setting) are specified with equal precision.

The paper also notes that the recaption explicitly includes "Preservation Statements" that instruct the generator about what must be maintained from the references, which helps prevent identity drift during the synthesis process.

Why recaptioning rather than direct image conditioning: The paper argues that raw reference images, while visually informative, have specific limitations that recaptioning addresses. Reference images may contain irrelevant background elements, watermarks, or compositional features that conflict with the user's prompt. A reference image of Dudoo sitting in a cup-shaped base might show the correct facial features but the wrong pose, wrong background, and wrong costume. By extracting only the identity-critical visual attributes into text form, the recaption allows the model to preserve what matters (the face, the defining features) while freely adapting composition, pose, and style to match the user's request.

The architectural synergy argument (Section 6.4). The paper makes an additional architectural claim about WHY recaptioning works well in a unified model versus decoupled systems. In Bagel-style models, visual inputs are tokenized through the dual VAE+ViT design. During recaptioning, both representations are available:

The ViT provides high-level semantic tokens suitable for reasoning about object identity, attribute relations, and scene composition.
The VAE provides low-level perceptual latents that retain fine-grained details like texture, material, and local geometry.

The paper argues that this dual representation enables recaptioning that is more precise and faithful than what a text-only LLM could produce by merely describing retrieved images. The LLM can only access the image through whatever textual description it generates or receives; the unified model has direct access to both semantic and perceptual representations of the reference images. The ablation results in Table 5 support this: removing ViT from the recaption stage (i.e., only having VAE latents) causes a large performance drop (Overall 73.2 → 61.4), and removing VAE (only having ViT tokens) also degrades performance (Overall 73.2 → 71.2), confirming that both representations contribute complementary information.

Stage 4: Generate — Evidence-Grounded Image Synthesis.

The final stage synthesizes the output image conditioned on the recaption and the selected visual anchors:

$y \sim p_\theta(y \mid c, \tau_v)$

This implements the conditional independence assumption from Equation 3: generation depends only on the refined specification $c$ and the visual anchors $\tau_v$ , NOT on the full upstream reasoning history ( $x$ , $g$ , $\tau_t$ ). The attention masking described in Section 3.4 (and Appendix B, Figure 6) enforces this at the architectural level: flow-matching tokens for the output image are restricted to attending only to recaption tokens and reference image tokens.

The paper identifies two advantages of this design:

Noise isolation: Noisy reasoning traces, redundant textual evidence, or irrelevant retrieval history cannot interfere with the continuous flow-matching process. The generator sees only the clean, structured recaption and the visual anchors.
Complementary conditioning signals: The recaption provides a unified, controllable textual description of the intended image (what should be depicted, how it should be composed, what style should be used), while the visual anchors stabilize identity realization (what the entity actually looks like). The recaption handles composition and style; the visual anchors handle identity fidelity. This complementary division of labor prevents the tension observed in Figure 2 where raw visual injection over-constrains composition while raw text injection fails to anchor identity.

The paper does not specify the exact number of flow-matching steps used at inference time, but given the base Bagel architecture and standard flow-matching practice, generation proceeds by starting from random noise in the VAE latent space and iteratively applying the predicted velocity field to converge to a clean latent, which is then decoded to an image via the frozen VAE decoder.

The FactIP Benchmark: Purpose-Built for World-Grounded Evaluation

Section 4.2 describes the construction of FactIP, a benchmark specifically designed to evaluate world-grounded image synthesis for long-tail concepts. While the technical approach section focuses on the system, understanding the benchmark design is relevant because it defines what "success" means for the technical approach.

Benchmark construction principles. FactIP samples are selected to satisfy four criteria:

Long-tail and knowledge-intensive: Successful generation requires external or internalized world knowledge beyond common popular concepts. The benchmark explicitly avoids entities that any reasonable T2I model would have memorized from pretraining data.
Visually grounded: Each target is associated with a clear visual identity that can be assessed against reference images. This is essential for evaluation — there must be a ground truth for what the entity actually looks like.
Difficult for memorization-only models: Strong performance cannot be achieved by relying solely on pretraining memorization or generic T2I priors. This distinguishes FactIP from generic image quality benchmarks.
Diverse across categories: The benchmark covers Character, Object, and Scene categories with 12 fine-grained subcategories (detailed in Table 7).

The benchmark contains 2,462 prompts in the full version, with a 500-sample FactIP-Mini subset that preserves the original category distribution. Critically, the benchmark is constructed from a disjoint subset of the IP pool used for training — benchmark instances are never seen during supervised fine-tuning.

Evaluation protocol. Each generated image is evaluated against the prompt and two ground-truth reference images along four dimensions:

Clarity (weight 0.05): Image sharpness, absence of artifacts, richness of visible details. Scored 0–10.
Content (weight 0.10): Faithfulness to the prompt's semantic elements and compositional requirements. Scored 0–10.
Aesthetics (weight 0.10): Overall visual quality including composition, lighting, color harmony, and stylistic appeal. Scored 0–10.
Relevance (weight 0.75): Identity consistency — whether the generated image preserves the defining visual attributes of the target IP as inferred from the reference images. Scored 0–10.

The overall score is:

$\text{Overall} = 10 \times (0.05 \cdot \text{Clarity} + 0.10 \cdot \text{Content} + 0.10 \cdot \text{Aesthetics} + 0.75 \cdot \text{Relevance})$

where the weights sum to 1.0 and the multiplication by 10 normalizes to a [0, 100] scale.

What this weighting reveals about the paper's priorities. The 0.75 weight on Relevance — 7.5× more than Content and Aesthetics, 15× more than Clarity — makes explicit that FactIP is designed to measure identity fidelity, not general image quality. A visually stunning image that depicts the wrong entity will score poorly; a technically adequate image that perfectly captures the target IP will score well. This weighting aligns with the paper's central claim that the bottleneck for long-tail generation is knowledge, not rendering quality.

The evaluation uses Seed2.0 as the expert evaluator (rather than GPT-4o used for other benchmarks), with the paper citing Seed2.0's "stronger multimodal judging capability." The evaluation prompt (Appendix E.2) instructs the judge to "use GT1 and GT2 jointly to infer the stable identity and attributes of the IP" and crucially, to "not penalize AS for differences that also vary between GT1 and GT2." This means if the two reference images already show variation in pose, lighting, or background, the model is not penalized for producing a different but equally valid interpretation — only identity-defining features that are stable across references are enforced.

Summary of Key Design Choices and Their Justifications

End-to-end unified architecture over decoupled pipeline: Eliminates cascading errors from API stitching, enables the VAE+ViT dual visual encoding to directly inform recaptioning, and allows joint optimization of reasoning, search, and generation under a single training objective.
Text-first, visual-second search ordering: Textual evidence resolves semantic ambiguity before visual search, enabling more precise and context-aware image retrieval. The ordering is deliberately supervised in training trajectories.
Top-2 image selection with four-dimensional quality filtering: Two images provide sufficient visual grounding without introducing conflicting information that would arise from larger sets. Quality filtering ensures the model learns to be selective about evidence rather than blindly trusting search results.
Recaptioning as an intermediate representation: Directly addresses the empirical finding that naive multimodal injection is suboptimal (Figure 2). Transforms heterogeneous evidence into a generation-optimized specification that disentangles identity-preserving constraints from scene-compositional constraints.
Generation-based verification via reject sampling for training data: Ensures that only trajectories producing verifyably faithful recaptions are included in training, filtering out ~68% of initial candidates (143K retained from 456K) and providing clean supervision.
Sequence packing with specialized attention masking: Enables efficient training on heterogeneous interleaved trajectories while enforcing the conditional independence assumption that generation should depend only on recaption and visual anchors, not on raw reasoning traces.
Special token reweighting (3×): Prioritizes learning correct structured output formats (tool calls, XML tags) that are essential for reliable agentic execution at inference time.
Constant learning rate schedule with 500-step warmup: Provides substantial plasticity for learning entirely new capabilities (tool use, evidence evaluation, recaptioning) from scratch during fine-tuning.
Frozen VAE for stable latent space: Prevents the flow-matching process from being destabilized by changes to the latent geometry during fine-tuning on a relatively small dataset.
Relevance-weighted evaluation (0.75 weight): Aligns benchmark scoring with the paper's claim that identity fidelity is the primary challenge for world-grounded generation, de-emphasizing generic image quality metrics.

4. Key Insights and Innovations

Innovation 1: World-Grounded Image Generation as Inference-Time Sequential Decision Making

The paper's most fundamental intellectual move is reframing what image generation is for knowledge-intensive prompts. The dominant paradigm in T2I — both in standard diffusion models and in unified multimodal models like Show-o, Janus, and the base Bagel model — treats generation as conditional sampling from a learned distribution p(y|x). The model's parametric memory θ is assumed to contain everything needed; the prompt is the sole interface. This is the "closed-book" assumption. The paper argues — and demonstrates — that this assumption catastrophically breaks for long-tail concepts, but more importantly, it proposes that the generative process itself should be reformulated as active, inference-time knowledge acquisition.

This is not simply "add retrieval to T2I." Prior retrieval-augmented approaches (Mind-Brush, World-to-Image, T2I-Copilot) do add retrieval, but they keep the generative model as a black box that receives augmented inputs. The generation step is still p(y|x, retrieved_stuff). The paper's reframing is more radical: generation becomes an interleaved trajectory over an augmented state space that includes the model's own assessment of what it doesn't know (g, the cognitive gap), the evidence it actively acquires (τ_t, τ_v), and an intermediate transformation of that evidence (c, the recaption) before any pixels are produced. The formal factorization in Equation 3 isn't just notation — it makes explicit that the model must reason about its own ignorance before it can compensate for it. This is a conceptual shift from "what should I generate?" to "what do I need to know to generate this correctly, and how do I find it?"

The significance beyond performance is that this reframing connects image generation to broader themes in AI that have been transformative in other domains: tool use, explicit uncertainty modeling, and agency. Just as LLM research discovered that reasoning improves when models externalize their thinking (chain-of-thought) and use tools (ReAct, function calling), this paper argues that T2I needs the same evolution — and provides the first end-to-end implementation where the same model handles all stages. The failure of Bagel-CoT (chain-of-thought without tool use, which reduces performance from 50.9 to 47.0 on FactIP, Table 1) is telling: internal reasoning without the ability to acquire external knowledge doesn't help, and may hurt by exposing knowledge gaps the model can't fill. This negative result suggests that the reframing isn't just additive (reasoning + generation), but requires the specific capability of evidence acquisition to be effective.

This is a fundamental reframing, not an incremental improvement. It changes the relationship between the model and the world from "memorize everything you might need during training" to "learn how to find what you need when you need it."

Innovation 2: Recaptioning as a Semantic Interface Mechanism for Multimodal Evidence

The paper identifies and solves a specific, subtle problem that prior work largely overlooked: retrieved evidence, even when correct, is in the wrong format for generation. Raw text search results contain redundant, non-visual information (social media posts, navigation headers, tangential background). Raw reference images carry irrelevant backgrounds, watermarks, and compositional elements that conflict with the user's specified scene. The counterintuitive finding in Figure 2 — that text + visual injection performs worse than visual injection alone (15.4 vs. 16.6 absolute improvement) — is not just an ablation result; it diagnoses a fundamental modality mismatch between what retrieval systems produce and what generators need.

Prior agentic T2I systems (GenArtist, World-to-Image) passed retrieved content directly to the generator, treating evidence integration as a formatting problem. The paper's innovation is recognizing that evidence integration is a representation problem: the model needs to understand the evidence (extracting identity-critical visual attributes from reference images) and transform it into a representation compatible with downstream generation (structured text that disentangles identity from composition). The recaption is not a prompt — it's a semantic interface specification that explicitly separates what must be preserved (facial features, defining visual traits) from what can be freely composed (pose, scene, style).

What makes this intellectually distinctive is that it exploits a capability that only unified models possess. In decoupled pipelines, the LLM planner can reason about text evidence but can only "see" images through whatever textual descriptions it generates. It cannot access low-level visual details like texture, facial structure, or fine-grained coloration. The recaptioning stage in Unify-Agent works because the model processes reference images through the dual VAE+ViT encoding: the ViT provides high-level semantic understanding (who is in the image, what attributes they have), while the VAE provides low-level perceptual detail (exactly what hair color, exactly what face shape). The paper explicitly argues this in Section 6.4: "generation can in fact improve understanding" — the generative priors embedded in the VAE latents serve understanding purposes during recaptioning.

The ablations in Table 5 substantiate this. Removing ViT from recaptioning drops the Overall score from 73.2 to 61.4 (an 11.8-point degradation, the largest single-component drop). Removing VAE causes a smaller but still meaningful drop to 71.2. These aren't just component importance scores — they demonstrate that the two visual representations contribute complementary and non-redundant information. ViT handles identity semantics; VAE handles perceptual fidelity. Without either, recaptioning quality degrades. In a decoupled system, the LLM has access to neither.

This is a fundamental architectural insight with implications beyond this specific pipeline. It suggests that unified understanding-generation models have an inherent advantage for tasks requiring fine-grained visual reasoning — not just because of parameter sharing, but because generative priors provide a representation of visual detail that pure vision-language models lack. Future work on multimodal agents for visual tasks (editing, design, visual QA about fine-grained attributes) should consider whether generating (via VAE latents) can improve understanding, rather than treating generation and understanding as separate capabilities.

Innovation 3: Cognitive Gap Detection as a Principled Trigger for Agentic Behavior

Most agentic systems follow a pattern of planning-then-executing — the agent decides on a course of action and carries it out. The paper introduces a more specific and theoretically motivated trigger: gap detection — the model explicitly assesses whether its parametric knowledge is sufficient before deciding whether to invoke external tools. This is not generic planning. It's a capability-self-assessment that conditions the entire subsequent pipeline on an estimated knowledge deficit.

Why is this distinctive? Prior retrieval-augmented generation (RAG) systems typically retrieve for every query, regardless of whether the parametric knowledge is sufficient. This is wasteful when the model already knows the answer and potentially harmful when retrieval introduces noise. Prior T2I systems with agentic components either always retrieve or use a separate classifier to decide whether to retrieve. The paper's approach — training the model to internalize this decision as part of the generative trajectory — is more elegant and more scalable: the same model that generates also decides when generation needs help.

The gap detection stage is represented as a set of missing knowledge units M(x) = {m_1, ..., m_K}, where each m_k identifies a specific attribute that is insufficiently specified. This is more granular than binary retrieval decisions. The model doesn't just decide "I need to search for this IP" — it identifies which attributes of the IP it needs to find: facial structure, hairstyle, costume details, color specifications. This granularity matters because it directly informs the subsequent search queries, making them more targeted. A query formulated from "I don't know what this character looks like" is less effective than one formulated from "I need to find the hair color, eye shape, and facial structure of this specific character."

The paper doesn't provide a direct ablation of gap detection (it's embedded in the full pipeline), but the failure of Bagel-CoT (which does reasoning without search) versus Unify-Agent (which does reasoning-then-search) on FactIP (47.0 vs. 73.2, Table 1) provides indirect evidence. Chain-of-thought without the ability to act on identified gaps doesn't help — the model can recognize it doesn't know something, but without tool access, that recognition is merely diagnostic. Gap detection only becomes useful when coupled with evidence acquisition, which is exactly what the sequential factorization in Equation 3 formalizes.

This is a significant refinement of agentic modeling rather than a fundamental new paradigm, but it addresses a specific weakness in prior work: the lack of principled triggers for tool invocation. By making the trigger an explicit, trainable component of the generative process, the paper shows that "when to act" can be learned alongside "how to act" and "what to generate."

Innovation 4: Verifier-Driven Evidence Quality Control as an Integral Training Signal

The paper introduces a specific mechanism for ensuring that agentic training data is reliable: generation-based verification via reject sampling during data construction, combined with four-dimensional image quality filtering during both training and inference. This isn't just data cleaning — it's a principled approach to the problem that agent trajectories are inherently noisy because search results are unpredictable.

The insight is that in an agentic pipeline, the supervision signal itself must be verified because the teacher model (Claude Opus 4.6) can produce plausible-looking but ultimately incorrect trajectories. A generated recaption might sound detailed and accurate while failing to produce a faithful image when actually used for generation. The paper's solution — generate an image from the recaption using Nano Banana Pro, check it against the ground-truth image using GPT-4o, and discard trajectories that fail after 5 attempts — closes the loop between data construction and the ultimate task. This reduces the initial 456K candidate concepts to 143K training trajectories, filtering out ~68% of the data.

Why is this conceptually significant? Most supervised fine-tuning for multimodal models uses human-verified or automatically generated labels without end-to-end task verification. The labels are assumed correct. The paper's approach acknowledges that for complex agentic tasks, the label itself is a hypothesis (this trajectory, if followed, should produce a faithful image) that needs empirical testing. This is methodologically similar to how RLHF uses preference labels verified by human raters, but applied to a different problem: verifying that a complex multi-step plan actually achieves its stated goal.

The image quality filtering (four-dimensional scoring by Gemini 3 Flash during data construction, and presumably by the trained model at inference) extends this principle to the evidence acquisition stage. Rather than trusting that search results are useful, the model is trained to evaluate evidence quality along multiple dimensions (identity consistency, subject salience, clarity, watermark cleanliness) and select only the best references. This addresses a key failure mode in retrieval-augmented systems: garbage-in, garbage-out. If the model retrieves a low-quality or incorrect reference image and bases its recaption on it, the generation will be wrong regardless of how good the generator is.

This is an incremental but practically important innovation. The individual components (reject sampling, image quality filtering) are not novel in isolation, but their integration into the trajectory construction pipeline — treating data quality as an end-to-end verification problem rather than a preprocessing step — addresses a real bottleneck in training agentic multimodal systems. The 68% data rejection rate suggests this filtering is not cosmetic; it's catching a substantial fraction of trajectories that would otherwise provide misleading supervision.

Innovation 5: The Architectural Synergy Claim — Generation Improves Understanding in Unified Models

The paper makes a specific, falsifiable architectural claim that goes beyond the immediate application to world-grounded synthesis: in a unified multimodal model with dual VAE+ViT visual encoding, the generative components (VAE latents) directly improve the understanding components' ability to produce faithful recaptions. This is not a claim about parameter sharing or multi-task learning — it's a claim about representation complementarity during inference.

The argument runs as follows: recaptioning requires the model to look at reference images and produce detailed textual descriptions of what it sees, specifically extracting identity-critical visual attributes. This is a visual understanding task. But the VAE latents — originally designed for image generation — provide low-level perceptual information (texture, color, material, local geometry) that the high-level ViT semantic tokens don't fully capture. When both representations are available during recaptioning, the model can reason about identity with greater fidelity. When either is removed (Table 5 recaption architecture ablations), performance degrades.

What makes this intellectually distinctive is that it inverts the typical relationship between generation and understanding. In most multimodal architectures, understanding is seen as supporting generation (the model understands what the prompt means, then generates). The paper argues that generation can support understanding — that having generative priors available during a reasoning task improves the quality of the reasoning itself. This is a specific, empirically testable claim that the paper supports with ablations but that has broader implications for UMM architecture design.

If the claim holds beyond this specific pipeline, it suggests that future unified models should make generative components available during understanding tasks, not just during generation. For example, a unified model performing visual question answering about fine-grained attributes might benefit from VAE-level representations even though no image is being generated. The current trend in unified models (Janus, Show-o, Emu3) is toward stronger separation of understanding and generation pathways; this paper provides evidence that tighter integration — specifically, making low-level perceptual latents accessible during understanding — may be beneficial.

This is a potentially fundamental architectural insight that needs further validation beyond the specific setting. The evidence is currently limited to the recaptioning task within Unify-Agent and to a single model family. But if the finding generalizes — if generative priors consistently improve fine-grained visual understanding — it would influence how the next generation of unified multimodal models is designed. The paper appropriately frames this as an observation that "helps explain why an end-to-end unified architecture is beneficial" (Section 6.4) rather than as a proven universal principle, but the empirical pattern is striking enough to warrant attention.

5. Experimental Analysis

Evaluation Methodology

Datasets. The paper evaluates on four benchmarks: FactIP (newly introduced, 2,462 prompts spanning 12 long-tail categories: Celebrity, Animation, Game, Comic, Mythology, Mascot, Animal, Food, Art, Toy, Landmark, Festival — with a 500-sample FactIP-Mini subset preserving category distribution), WiSE (world-knowledge-informed semantic evaluation across cultural, time, space, biology, physics, and chemistry dimensions), KiTTEN (knowledge-intensive evaluation of image generation on visual entities, spanning 8 categories with text alignment and entity alignment scores), and T2I-FactualBench (three-tiered factual accuracy benchmark: Single Knowledge Concept Memorization [SKCM], Instantiation [SKCI], and Multiple Concept Composition with Interaction [MKCC]). FactIP is constructed from a disjoint subset of the IP pool used for training — benchmark instances were never seen during fine-tuning.
Base model(s). Unify-Agent is built on Bagel-14B (Deng et al., 2025a), a unified multimodal model with a Mixture-of-Transformers architecture, SigLIP-SO400M-14 ViT encoder with NaViT, and FLUX VAE (frozen during Unify-Agent training). The choice is motivated by Bagel's native support for both autoregressive text generation and flow-matching image synthesis within a shared backbone, which enables end-to-end training of the full agentic pipeline. The paper does not experiment with alternative base architectures (e.g., Janus, Emu3) as the underlying UMM for Unify-Agent.
Metrics. FactIP uses a weighted multi-dimensional protocol with Seed2.0 as evaluator: Clarity (weight 0.05), Content (0.10), Aesthetics (0.10), and Relevance (0.75), each scored 0–10, with the overall score computed as 10 × (0.05×Clarity + 0.10×Content + 0.10×Aesthetics + 0.75×Relevance) normalized to [0, 100]. The heavy Relevance weighting (75% of the total) reflects the benchmark's focus on identity fidelity rather than generic image quality. WiSE uses GPT-4o as evaluator, scoring Consistency, Realism, and Aesthetic Quality each on a 0–2 scale, combined into a WiScore with weights (0.7, 0.2, 0.1). KiTTEN uses GPT-4o to independently score Text Alignment and Entity Alignment on 1–5 scales without fusing them into a single metric. T2I-FactualBench uses a multi-round VQA protocol with GPT-4o: Concept Factuality (binary scores on Shape, Color, Texture, Feature Details averaged across concepts), Instantiation Completeness (binary presence check), and Composition Factuality (four binary dimensions for MKCC).
Baselines. The paper benchmarks against three categories: (1) Commercial models: GPT-Image-1, GPT-Image-1.5, Seedream-4/4.5/5, Qwen-Image-2.0-Pro, Nano Banana/Pro/2, DALL-E 3 (Betker et al., 2023). (2) Generation-only open-source models: FLUX.1-schnell/dev, FLUX.2-dev, SD-3-medium/large, SDXL-base-0.9, SD-3.5-medium/large, Playground-v2.5 (Li et al., 2024), Z-Image (Team, 2025c), Qwen-Image (Wu et al., 2025a), Pixel-Art-XL. (3) Unified MLLMs: Janus-1.3B, Janus-Pro-7B (Chen et al., 2025d), Emu3/Emu3.5 (Cui et al., 2025), Echo-4o (Ye et al., 2025), Hunyuan-Image-3.0 (Cao et al., 2026), BLIP3o-8B, MetaQuery-XL, UniWorld-V1, vila-u-7b-256, Bagel (Deng et al., 2025a), and Bagel-CoT (Bagel with chain-of-thought reasoning but no external tool use). On KiTTEN, additional baselines include SD, Flux, Custom-Diff, Imagen, DreamBooth, Instruct-Imagen, and Imagen-3 from prior work.
Generation budget / compute accounting. The paper does not conduct a FLOPs-matched comparison or measure inference compute in a standardized unit (unlike the reference example paper's "generations" budget). For search-based agentic behavior, each trajectory involves exactly one round of text search and one round of image search (with top-2 selection after quality filtering). The recaptioning and generation stages are single-pass. There is no systematic sweep of compute budgets (e.g., varying the number of search rounds, the number of reference images, or the flow-matching steps) to characterize compute-optimal scaling. This is a notable difference from test-time compute scaling work and represents a limitation in understanding how Unify-Agent's performance varies with inference budget.
Cross-validation / statistical protocol. For FactIP, the benchmark is separated from training data by construction (disjoint subsets). Evaluation uses the FactIP-Mini 500-sample subset with the fixed Seed2.0 evaluator. There is no mention of cross-validation, multiple evaluation runs, or confidence intervals. For WiSE, KiTTEN, and T2I-FactualBench, the paper follows each benchmark's official evaluation protocol exactly, including their designated evaluators (GPT-4o for all three) and scoring mechanisms. No statistical significance testing or error bars are reported across any benchmark, which is standard for MLLM-as-a-judge evaluation in this subfield but limits the ability to assess whether observed differences (e.g., Unify-Agent's 73.2 vs. Bagel's 50.9 on FactIP) are robust to evaluator variation.

Main Quantitative Results

FactIP Benchmark: World-Grounded Synthesis of Long-Tail Concepts

Headline result: Unify-Agent achieves an Overall score of 73.2 on FactIP, surpassing its base model Bagel by 22.3 points (50.9 → 73.2) and outperforming all generation-only baselines and all other unified MLLMs. This is the largest margin of improvement in the unified MLLM category (Table 1).

Comparison within unified MLLMs: Bagel-CoT (chain-of-thought reasoning without tool use) scores 47.0 — actually worse than vanilla Bagel (50.9), demonstrating that internal reasoning without external evidence access does not help and may expose knowledge gaps that degrade generation. The next-best unified model is Emu3.5 at 57.2, followed by Hunyuan-Image-3.0 at 53.4. Unify-Agent's 73.2 represents a 16-point gap over the second-best open-source unified model. The Relevance dimension shows the most dramatic improvement: Unify-Agent scores 72.4 (averaged as (67.3 + 71.8 + 78.2) × 0.75 / 0.75 = 72.4 weighted by category) compared to Bagel's 44.9 — a 27.5-point increase, confirming that the gains come primarily from improved identity fidelity rather than generic image quality.

Comparison with generation-only models: The gap is even larger. FLUX.1-dev scores 28.9 Overall, SD-3.5-large scores 27.5, and Qwen-Image scores 55.4. Even FLUX.2-dev (56.3) and Z-Image (54.2) — the strongest generation-only models — trail Unify-Agent by ~17–19 points. On Relevance specifically, FLUX.1-dev scores 17.0 overall versus Unify-Agent's 72.4 (the paper reports per-category Relevance in Table 1: Character 67.3, Object 71.8, Scene 78.2; the overall Relevance weighted by category distribution is approximately 72.4 based on the individual scores).

Comparison with commercial models: Unify-Agent's 73.2 places it between Seedream-4.5 (82.0) and Seedream-5 (87.3) but ahead of GPT-Image-1.5 (69.9), Qwen-Image-2.0-Pro (71.1), and Nano Banana-Pro (66.7). Nano Banana-2 leads at 88.5. This positions Unify-Agent as competitive with mid-tier commercial systems while being entirely open-source. The paper notes Unify-Agent "demonstrates world knowledge capabilities approaching those of leading commercial models" (Section 1), though the 15.3-point gap to Nano Banana-2 indicates substantial room for improvement.

Category-level analysis (Table 8, Figure 7b): Unify-Agent shows balanced performance across categories but with notable variation. Within Character subcategories: Celebrity 71.9, Animation 71.7, Game 68.0, Comic 69.0, Mythology 76.5, Mascot 75.7. Within Object: Animal 75.5, Food 80.2, Art 65.3, Toy 76.3. Within Scene: Landmark 70.9, Festival 74.0. The strongest performance is on Food (80.2) and Mythology (76.5); the weakest on Art (65.3) and Game (68.0). The paper does not analyze why certain categories are harder — possible factors include the quality of available reference images, the distinctiveness of identity-defining features, or category-specific biases in the teacher model used for data construction.

WiSE Benchmark: World Knowledge Across Domains

Headline result: Unify-Agent achieves an Overall WiScore of 0.77, the highest among all unified MLLMs and competitive with some commercial models (Table 2). This represents a 0.25-point improvement over Bagel (0.52) and a 0.07-point improvement over Bagel-CoT (0.70).

Domain-level analysis: Unify-Agent leads unified models in Cultural (0.82), Biology (0.72), and Chemistry (0.70), while scoring competitively on Time (0.75), Space (0.74), and Physics (0.73). The Cultural score of 0.82 is the highest among all open-source models and approaches commercial performance (Nano Banana: 0.89, Nano Banana-Pro: 0.89, GPT-Image-1: 0.81). The Chemistry score of 0.70 is notably strong — the next-best unified model is BAGEL+CoT at 0.58, and most unified models score below 0.40 on Chemistry. This suggests Unify-Agent's evidence acquisition is particularly effective for domains where textual knowledge (chemical facts, biological taxonomy) provides strong grounding for visual synthesis.

Comparison with generation-only models: The gap is substantial. The best open-source generation-only model is Qwen-Image at 0.62, while FLUX.1-dev scores 0.50 and SD-3.5-large scores 0.46. Unify-Agent's 0.77 represents a 0.15-point lead over the best generation-only model and a 0.31-point lead over FLUX.1-dev. The gap is largest on Cultural (0.82 vs. Qwen-Image's 0.62) and Chemistry (0.70 vs. Qwen-Image's 0.40).

Comparison with commercial models: Nano Banana leads at 0.89, with Nano Banana-Pro at 0.87 and GPT-Image-1 at 0.80. Unify-Agent (0.77) trails the top commercial systems by 0.12–0.20 points but outperforms GPT-Image-1 on Cultural (0.82 vs. 0.81) and Chemistry (0.70 vs. 0.74 — actually slightly lower; the paper's claim that Unify-Agent "particularly excels in cultural, biological and chemistry knowledge" is supported for Cultural and Biology but less so for Chemistry relative to GPT-Image-1's 0.74, though Unify-Agent's 0.70 does lead all open-source models by a wide margin).

Interpretation relative to FactIP: The WiSE improvement over Bagel (+0.25) is proportionally smaller than the FactIP improvement (+22.3 points). This likely reflects WiSE's broader scope — it tests general world knowledge (physics, chemistry, biology, cultural facts) that Bagel may already possess to some degree parametrically, whereas FactIP explicitly targets long-tail IPs where parametric knowledge is near-zero. The agentic pipeline provides larger relative gains when the knowledge gap is absolute rather than partial.

KiTTEN Benchmark: Fine-Grained Entity Alignment

Headline result: Unify-Agent achieves an Overall score of 4.08 on KiTTEN, establishing a new state of the art among both generation-only and unified MLLM approaches, and surpassing the strong commercial baseline Imagen-3 (3.50) by 0.58 points (Table 3). Text Alignment is 4.22 and Entity Alignment is 3.93 — both the highest in their respective categories.

Category-level analysis: Unify-Agent leads or ties for the lead in 14 of 16 sub-scores (8 categories × 2 dimensions each). Notable performances include: Aircraft (Text 4.03, Entity 3.59), Vehicle (4.54, 4.09), Cuisine (4.36, 4.13 — both highest overall), Flower (4.28, 4.16), Insect (4.17, 3.97), Landmark (3.99, 3.76), Plant (4.46, 3.81), Sport (3.95, 3.92). The only sub-scores where Unify-Agent does not lead are Aircraft Entity (Instruct-Imagen scores 3.88 vs. Unify-Agent's 3.59) — an interesting anomaly the paper does not discuss.

Comparison with Bagel variants: Bagel scores 3.04 Overall (Text 3.44, Entity 2.64). Bagel-CoT scores 2.84 Overall (Text 3.66, Entity 2.02) — again, chain-of-thought without tool access reduces entity alignment performance (2.64 → 2.02), consistent with the FactIP finding that more explicit reasoning about missing knowledge without the ability to acquire it can actually degrade faithfulness. Unify-Agent improves Entity Alignment from Bagel's 2.64 to 3.93 (+1.29, a 49% relative improvement), demonstrating that external visual evidence acquisition directly addresses the entity fidelity bottleneck.

Comparison with generation-only models: The best generation-only model is Imagen-3 at 3.50, followed by Instruct-Imagen at 3.17. Unify-Agent's 4.08 represents a substantial lead. Entity Alignment shows the widest gap: Unify-Agent's 3.93 vs. Imagen-3's 2.83 (+1.10). This is the dimension where external visual evidence should matter most — entity alignment requires faithful reproduction of specific visual entities, which is exactly what the image search and recaption pipeline is designed to enable.

T2I-FactualBench: Factual Accuracy Across Difficulty Levels

Headline result: Unify-Agent achieves top scores among unified models on SKCI (77.4) and MKCC (71.5), and records an SKCM Concept score of 69.2 that compares favorably against DALL-E 3 (55.5) (Table 4).

Three-tier progression: The results show an interesting pattern as task complexity increases. On SKCM (single knowledge concept memorization — can the model recall what a concept looks like?), Unify-Agent scores 69.2 on Concept, substantially higher than Bagel (31.8) and DALL-E 3 (55.5). On SKCI (instantiation — can the model adapt a concept to varied conditions like different actions or scenes?), Unify-Agent scores 77.4 Overall (Concept 75.3, Instantiation 79.6), compared to Bagel's 58.2 and SD-3.5's 67.9. On MKCC (multiple concept composition with interaction — the hardest tier requiring multiple concepts to be combined coherently), Unify-Agent scores 71.5 Overall (Concept 76.1, Instantiation 64.8, Composition 73.6), compared to Bagel's 66.3 and DALL-E 3's 75.7.

The pattern is revealing: Unify-Agent's advantage over Bagel is largest on SKCM Concept (+37.4 points: 31.8 → 69.2), suggesting that for pure concept memorization, the agentic pipeline compensates most effectively for missing parametric knowledge. The advantage narrows on MKCC (+5.2 points: 66.3 → 71.5), where composition and interaction constraints become dominant — these are capabilities that depend more on the base model's generation quality than on external knowledge. Notably, DALL-E 3 leads on MKCC (75.7), suggesting that commercial models maintain an advantage on complex compositional generation even if they underperform on long-tail concept knowledge.

Comparison with generation-only models: SD-3.5 scores 67.9 on SKCI and 67.9 on MKCC, while FLUX.1-dev scores 56.5 and 58.3 respectively. Unify-Agent leads both by substantial margins, though DALL-E 3 remains competitive or ahead (SKCI 80.5, MKCC 75.7). The paper does not report Nano Banana or other commercial models on T2I-FactualBench besides DALL-E 3, limiting the completeness of the commercial comparison on this particular benchmark.

Ablation Studies and Robustness Checks

All ablations are reported in Table 5 on the FactIP benchmark against the vanilla Bagel baseline (Overall 50.9). Numbers in parentheses show change relative to Bagel; red indicates improvement, green indicates degradation.

Pipeline component ablations (text search, image search, recaptioning): Removing text search drops Overall from the full model's 73.2 to 65.4 (−7.8 from full, +14.5 over Bagel), with the largest impact on Content (75.2 → 70.9). Removing image search causes a more severe drop to 56.2 (−17.0 from full, only +5.3 over Bagel), with Relevance collapsing from 72.4 to 50.8 — a 21.6-point drop that confirms visual evidence as the primary driver of identity fidelity. Removing recaptioning drops Overall to 62.9 (−10.3 from full), with Clarity (91.2 → 83.0) and Aesthetics (85.2 → 74.5) also degrading substantially. This is the critical ablation: it demonstrates that raw evidence injection (which is what "w/o Recaption" effectively tests — passing retrieved text and images directly to generation) is substantially worse than the recaption-mediated approach, validating the paper's central design claim.
Constraint-level ablations (identity-preserving vs. scene-compositional): Removing identity-preserving constraints from the recaption (i.e., recaptioning without explicit facial feature and defining-attribute preservation) drops Overall to 67.7 (−5.5 from full), with Relevance declining to 65.9 (−6.5). Removing scene-compositional constraints drops Overall to 68.2 (−5.0), with Content declining to 70.8 (−4.4) and Aesthetics to 80.7 (−4.5). Both constraint types contribute meaningfully but identifiably different dimensions: identity constraints primarily affect Relevance (identity fidelity), while scene constraints primarily affect Content and Aesthetics (compositional quality). The fact that removing either degrades Overall by ~5 points confirms that the disentanglement is not redundant — both constraint types are necessary.
Recaption architecture ablations (VAE removal, ViT removal): Removing VAE from the recaption stage (i.e., the model can only use ViT semantic tokens when interpreting reference images) drops Overall to 71.2 (−2.0 from full). Removing ViT causes a much larger drop to 61.4 (−11.8 from full), with Relevance declining sharply from 72.4 to 58.7 (−13.7). This is the paper's key architectural evidence for the claim that high-level semantic understanding (ViT) is critical for accurate evidence interpretation during recaptioning, while low-level perceptual features (VAE) provide complementary but less essential detail. The paper frames this as evidence that "generation can improve understanding" — VAE latents serve an understanding role during recaptioning.
Bagel-CoT as implicit ablation of tool use: Table 1 shows Bagel-CoT scoring 47.0 on FactIP, below vanilla Bagel's 50.9. While not presented as a formal ablation in Table 5, this result demonstrates that adding chain-of-thought reasoning to the base model without the ability to acquire external evidence actually degrades performance. This is consistent across benchmarks: Bagel-CoT scores lower than Bagel on FactIP (47.0 vs. 50.9), KiTTEN (2.84 vs. 3.04), and shows mixed results on WiSE (0.70 vs. 0.52 — an improvement, suggesting WiSE's world knowledge benefits from reasoning even without retrieval). The paper does not analyze why CoT helps on WiSE but hurts on FactIP and KiTTEN, but a plausible explanation is that WiSE tests broader world knowledge where parametric memory is partially sufficient (reasoning helps connect known facts), while FactIP and KiTTEN test entity-specific visual knowledge where parametric memory is near-zero (reasoning only exposes the gap without filling it).
Ablations NOT performed that would strengthen the paper: The paper does not ablate (1) the number of reference images (is top-2 optimal vs. top-1 or top-3?), (2) the sequential text-then-visual ordering (what if visual search is performed first or simultaneously?), (3) the four-dimensional image quality filtering (what if unfiltered retrieval is used?), (4) the teacher model choice for trajectory construction (would a weaker teacher produce significantly worse trajectories?), (5) the reject sampling threshold (how does performance vary with 3 vs. 5 vs. 10 retry attempts?), (6) the recaption format (would a different structured specification format work equally well?), or (7) the dual ViT+VAE encoding vs. using only one of them for ALL visual processing (not just during recaptioning). These are not minor omissions — several represent critical design choices that the paper presents as motivated but does not empirically validate through ablation.

Critical Assessment

Does the paper demonstrate that the agentic pipeline causes the improvement, or merely that fine-tuning on a large curated dataset improves performance?

This is the most important question for evaluating the paper's central claim. Unify-Agent differs from Bagel in two substantial ways: (1) it is fine-tuned on 143K high-quality, purpose-built agent trajectories, and (2) it has access to an agentic inference pipeline (search tools, recaptioning). The experiments cannot fully disentangle these. The "pipeline ablations" in Table 5 remove specific components at inference time from the already-trained model — they test whether a trained Unify-Agent with text search disabled, or image search disabled, or recaptioning disabled performs worse. This demonstrates that the trained components are used and useful at inference, but it does not demonstrate that training on agent trajectories is necessary versus, say, training on the same 143K examples reformatted as standard prompt-image pairs with ground-truth reference images embedded in context. A critical missing baseline is: Bagel fine-tuned on the same 143K data but without the agentic structure — i.e., directly conditioning on the two reference images + a human-written or auto-generated detailed caption, without the intermediate search and recaption steps. If this baseline performed comparably to Unify-Agent, it would suggest the gains come primarily from training data quality and scale rather than from the agentic formulation. The paper provides no such baseline.

Does the evidence support the claim that "generation improves understanding" (Section 6.4)?

The paper supports this claim primarily through the recaption architecture ablations (Table 5: VAE removal drops Overall by 2.0 points, ViT removal drops it by 11.8 points). This demonstrates that both visual representations contribute to recaptioning quality, and that the VAE latents (a generative component) provide useful information for the understanding task of recaptioning. However, the claim "generation improves understanding" implies a directionality and mechanism that the ablation does not fully establish. It shows that generative features (VAE latents) are useful for understanding, but not that the act of generation or the generative training objective causally improves understanding capabilities. An alternative interpretation is simply that VAE latents capture complementary visual information that happens to be useful — not that generation and understanding are mutually reinforcing in any deep sense. A stronger test would be: does a model trained with the generation objective (flow matching) produce better recaptions than a model with identical architecture trained only on understanding objectives (even if it still has access to VAE-like features)? The paper does not run this comparison. The claim is intriguing and plausible but should be treated as a suggestive observation rather than a rigorously demonstrated causal relationship.

How robust are the results to evaluator choice?

All benchmarks use MLLM-as-a-judge (GPT-4o for WiSE, KiTTEN, T2I-FactualBench; Seed2.0 for FactIP). The paper does not report human evaluation, inter-evaluator agreement, or sensitivity to evaluator prompts. For FactIP specifically, the 0.75 weight on Relevance means that small variations in how Seed2.0 interprets "identity consistency" could substantially change Overall scores. The evaluation prompt (Appendix E.2) includes nuanced instructions ("do not penalize AS for differences that also vary between GT1 and GT2"), which requires the evaluator to perform comparative visual reasoning. No calibration of Seed2.0 against human judgments on this specific task is reported, nor is there analysis of whether Seed2.0 and GPT-4o (used for other benchmarks) would produce consistent rankings if both evaluated FactIP. This is not a criticism unique to this paper — the entire subfield is converging on MLLM-as-a-judge — but it means reported scores should be interpreted as relative rankings under a specific evaluator rather than absolute measures of generation quality.

Are the benchmarks genuinely testing world knowledge grounding, or are they testing retrieval quality?

FactIP, KiTTEN, and WiSE all involve prompts where the target entity is explicitly named and reference images are available. Unify-Agent's pipeline searches for those entities and uses the retrieved references. If the search system returns high-quality reference images, the recaption will be accurate; if search fails, the generation will suffer regardless of the model's reasoning capabilities. The paper does not ablate the search backend quality (e.g., comparing Google Search vs. Bing vs. a curated database) or report retrieval success rates. On benchmarks where the target entity is clear and well-documented (most of FactIP's curated list), search is likely to succeed; the pipeline's contribution may be more about effectively using retrieved evidence than about the act of retrieval itself. The true test of agentic capability would be prompts where the entity is ambiguous, the search results are noisy or conflicting, and the model must reason about which evidence to trust — a more realistic open-world setting that FactIP, being highly curated, may not capture.

What is the significance of the 68% data rejection rate?

The paper reports that 143K trajectories were retained from an initial pool of 456K — a 68.6% rejection rate. This is presented as evidence of quality filtering, but it also indicates that the trajectory construction pipeline (Claude Opus 4.6 for research + Nano Banana Pro for verification) produces trajectories that fail to generate faithful images in more than two-thirds of cases. This high failure rate raises questions about the teacher model's reliability and the difficulty of the task. It also means Unify-Agent is trained on a highly selected subset of "easy" trajectories where the pipeline demonstrably works. Whether the model generalizes to more challenging cases where the teacher pipeline would have failed is untested. The FactIP benchmark, being constructed from the same IP pool but with rigorous manual filtering (Section 4.2.1 mentions "manual filtering" and removal of "severe information noise, unclear instructions, low-quality reference images, or highly ambiguous identities"), may similarly represent a curated subset where the task is relatively well-defined. The reported performance on FactIP may thus be an upper bound on real-world performance where user prompts are less clean and reference-quality images are harder to find.

Missing baselines of practical importance:

Several comparisons that would clarify the value of the end-to-end agentic approach are absent: (1) Bagel + a simpler retrieval pipeline (e.g., Google Image Search for the entity name, top-2 images, no quality filtering, no recaptioning — just condition directly on those images and the prompt); (2) Unify-Agent without the generation-based verification during training (to test whether the reject sampling contributes beyond standard SFT); (3) A variant where the recaption is generated by an external LLM (e.g., GPT-4o) given the prompt and retrieved images, then fed to Bagel for generation — this would test whether the recaptioning must be done by the same unified model or whether an external strong LLM could perform equally well, which would challenge the architectural synergy claim.

Does the paper demonstrate that its approach works for "open-world" generation as claimed?

The paper's title and introduction emphasize "world-grounded" and "open-world" image synthesis, but the evaluation is confined to benchmarks where correct answers are well-defined and ground-truth reference images exist. For genuinely open-world generation — where the user's prompt references a novel combination of concepts, a fictional entity, or a scene for which no reference images exist — it is unclear how the pipeline would behave. The gap detection stage might correctly identify missing knowledge, but if search returns no relevant results or only tangentially related ones, the recaption would need to either hallucinate (defeating the purpose) or signal inability. The paper provides no analysis of failure modes or the model's behavior when search fails. The qualitative examples in Appendices F and G show successful cases; no systematic failure analysis (e.g., "on X% of FactIP prompts, search returned incorrect or irrelevant reference images, leading to Y% degradation") is presented.

Summary assessment: The experiments convincingly demonstrate that Unify-Agent substantially outperforms its base model and other open-source unified models on benchmarks designed to test knowledge-intensive generation. The ablation studies in Table 5 provide good evidence for the contribution of individual pipeline components at inference time. However, the experiments do not fully isolate the effect of the agentic design from the effect of high-quality training data, do not establish robustness to search quality or evaluator choice, and do not test generalization beyond curated benchmarks to truly open-world settings. The architectural synergy claim (generation improves understanding) is supported by suggestive ablations but not by the stronger causal tests that would definitively establish it. The paper's core contribution — that an end-to-end unified agentic pipeline can bridge the knowledge gap for long-tail visual concepts — is supported, but the boundary conditions (when does it fail? how dependent is it on retrieval quality? how much of the gain is from data vs. architecture?) remain largely unexplored.

6. Limitations and Trade-offs

6.1 Difficulty Estimation and Data Filtering Costs Are Unaccounted For

The assumption or constraint. The entire Unify-Agent pipeline depends on the ability to construct high-quality agent trajectories for supervised fine-tuning, which in turn requires (1) a curated pool of 456K long-tail IPs with manually verified metadata and seed images, (2) a strong teacher agent (Claude Opus 4.6) to synthesize multimodal research traces, (3) Gemini 3 Flash for image quality scoring, (4) Nano Banana Pro for generation-based verification, and (5) GPT-4o as a multimodal judge. The paper acknowledges that 143K trajectories were retained from the initial 456K pool — a 68.6% rejection rate (Section 4.1.3) — but does not account for the computational or financial cost of this data construction pipeline in any reported metric.

The consequence. The headline results (73.2 on FactIP, 4.08 on KiTTEN, 0.77 on WiSE) represent the performance achievable given access to a high-quality trajectory dataset constructed with multiple frontier commercial models. A practitioner attempting to replicate this work for a new domain or model family would need to reproduce the entire data pipeline — IP curation, manual verification, teacher agent trajectory synthesis, quality scoring, reject sampling, and judge evaluation — which requires API access to multiple proprietary systems (Claude Opus 4.6, Gemini 3 Flash, Nano Banana Pro, GPT-4o) and substantial human annotation effort. The paper provides no analysis of how performance degrades if weaker or open-source models are substituted, if fewer IP categories are covered, or if the reject sampling threshold is relaxed. This means the reported results should be understood as a demonstration of what is achievable given a particular data construction budget, not as a cost-calibrated claim about the general effectiveness of the approach.

What evidence exists in the paper. The 68.6% rejection rate is stated in Section 4.1.3 but never cost-analyzed. The paper notes that "after this filtering process, we obtain a final dataset containing 456K examples" (Section 4.1.1) and that the final training set is 143K trajectories (Section 4.1.3), but the cost in API calls, human annotation hours, or GPU compute to bridge the 456K → 143K gap is never quantified. The ReST^EM experiment (Appendix K), which attempted to optimize the revision model but degraded performance, hints that data construction methodology is sensitive to specific choices — yet no ablation studies vary the teacher model, the judge model, or the reject sampling threshold to characterize this sensitivity.

Mitigation status. Not addressed. The paper treats the data construction pipeline as infrastructure rather than as a cost center. There is no discussion of how to reduce data construction costs, whether cheaper models could substitute, or how the approach scales to domains without well-curated IP databases. A practitioner reading the paper has no way to estimate the total cost of replicating the training pipeline.

6.2 Hard Problems Remain Essentially Unsolved

The assumption or constraint. The paper demonstrates strong performance on knowledge-intensive concepts where (1) the target entity is clearly identifiable from the prompt, (2) external search systems return relevant, high-quality reference images, and (3) the base model has sufficient generative capability to compose the scene once identity is grounded. However, the paper provides no systematic analysis of failure modes when these conditions are violated, and the evaluation benchmarks (FactIP, KiTTEN, WiSE) are constructed to have ground-truth reference images available for scoring — they do not test genuinely open-world generation where no reference exists.

The consequence. A practitioner deploying Unify-Agent in a real-world setting would encounter prompts where search fails (no relevant images found), returns incorrect images (a different character with a similar name), or returns low-quality images (blurry, watermarked, off-angle). The paper provides no characterization of how the model behaves in these failure modes: does it recognize that search failed and refuse to generate? Does it hallucinate based on parametric knowledge? Does the quality filtering reject all candidates and leave the pipeline stranded? Similarly, for prompts involving novel combinations of concepts (e.g., "Dudoo in the style of a Baroque oil painting emerging from earthquake rubble," as in Figure 15), the model must simultaneously preserve identity fidelity from reference images while executing artistic transformations — the paper shows one successful qualitative example but no quantitative analysis of how often such compositional generalization succeeds versus produces artifacts or identity drift.

What evidence exists in the paper. The paper provides only successful qualitative examples (Appendices F and G) — no systematic failure analysis. The FactIP benchmark's per-category scores (Table 8) reveal substantial variation: Unify-Agent scores 80.2 on Food but only 65.3 on Art and 68.0 on Game. The paper does not analyze why certain categories are harder. On the T2I-FactualBench MKCC tier (Table 4), Unify-Agent's Composition score of 73.6 trails DALL-E 3's 75.7, suggesting that even with evidence grounding, complex multi-concept compositional generation remains challenging. There is no reported metric for "search failure rate," "reference image rejection rate," or "identity drift rate under compositional prompts."

Mitigation status. Not addressed. The paper focuses exclusively on demonstrating success cases and aggregate benchmark scores. Section 7 acknowledges that "our current pipeline is still limited to a relatively shallow one-pass workflow" but frames this as a limitation of iterative behavior rather than of robustness to evidence quality. The absence of failure analysis makes it impossible for a practitioner to estimate the system's reliability in deployment or to identify which types of prompts are safe versus which are likely to fail.

6.3 The Agentic Pipeline Is Single-Pass and Non-Iterative

The assumption or constraint. Unify-Agent executes exactly one round of text search, one round of image search (with top-2 selection), one recaptioning step, and one generation step. The paper explicitly acknowledges this in Section 7:

"our current pipeline is still limited to a relatively shallow one-pass workflow, rather than more general iterative agent behaviors such as interleaved text-image search, reflection, and replanning, which are crucial for harder open-world tasks such as travel planning or academic report generation."

The model cannot, for example, generate a preliminary image, evaluate whether identity fidelity is sufficient, and perform additional targeted search to refine specific attributes that were captured incorrectly. It cannot search for different types of evidence (e.g., detailed costume specifications, lighting references for a particular time period) in separate rounds. It cannot replan if the initial search returns ambiguous or conflicting results.

The consequence. The single-pass design fundamentally limits the system's ability to handle ambiguous prompts (where the model needs to disambiguate through iterative search) or complex compositional scenes (where different aspects of the scene might benefit from different evidence sources). For example, a prompt like "Scottie Pippen wearing his Chicago Bulls jersey, driving to the basket in a packed 1990s playoff game, in the style of a Baroque oil painting" requires (1) identity evidence for Pippen, (2) identity evidence for the Bulls jersey and number, (3) scene composition evidence for a driving layup, (4) style evidence for Baroque painting techniques. A single-pass search and recaption must handle all of these simultaneously; an iterative agent could tackle them sequentially, verify intermediate results, and adjust queries based on what was already found. The paper's qualitative examples show the model handling complex prompts, but the single-pass constraint means that if any one evidence dimension fails, there is no mechanism for recovery.

What evidence exists in the paper. The paper acknowledges this limitation explicitly in Section 7. The consistent gap between Unify-Agent and top commercial models on composition-heavy benchmarks — DALL-E 3 leads Unify-Agent on T2I-FactualBench MKCC (75.7 vs. 71.5, Table 4), and Nano Banana-2 leads on FactIP (88.5 vs. 73.2, Table 1) — may partially reflect the inability to iteratively refine evidence acquisition. However, the paper does not ablate the number of search rounds or compare single-pass vs. multi-pass variants, so the causal contribution of the single-pass constraint to the performance ceiling cannot be estimated from the reported experiments.

Mitigation status. The paper identifies this as future work in Section 7: "we plan to validate these findings on stronger unified backbones and extend the framework toward more capable multimodal agents that can support longer-horizon planning, repeated search, and adaptive reasoning." No partial mitigation is implemented or evaluated.

6.4 No Accounting for Latency, Throughput, or Deployment Complexity

The assumption or constraint. The paper measures performance solely in terms of benchmark scores and makes no attempt to characterize the inference-time cost of the agentic pipeline. At inference time, Unify-Agent must execute: (1) structured prompt understanding and gap detection (one autoregressive generation pass), (2) text search query formulation and external search API call (network latency + search engine processing time), (3) text evidence processing (another generation pass), (4) visual search query formulation and external search API call, (5) image quality scoring (one generation pass per candidate image, for n candidates), (6) recaptioning (a generation pass over the full context including reference images' ViT+VAE tokens), and (7) flow-matching image generation (an unknown number of iterative denoising steps). The paper provides no latency measurements, no throughput analysis, and no comparison of total wall-clock time versus the baseline models it outperforms on accuracy.

The consequence. The accuracy improvements reported in Tables 1–4 come with an unquantified — and likely substantial — increase in inference cost and latency. A single Unify-Agent generation cycle involves multiple serial stages (text search must complete before visual search; search results must be processed before recaptioning; recaptioning must complete before generation), each of which blocks progress. The baseline models (Bagel, FLUX.1-dev, SD-3.5-large) generate images in a single forward pass (or a fixed number of diffusion steps), making them dramatically faster per prompt. A practitioner choosing between deploying Unify-Agent versus a simpler model with a larger parameter count cannot make an informed cost-benefit tradeoff without knowing whether the 22.3-point FactIP improvement over Bagel (Table 5) requires 2×, 10×, or 100× the inference time and compute.

This tradeoff is particularly acute for the paper's motivating use cases. If a content creator is generating concept art interactively, a 30-second agentic pipeline may be unacceptable regardless of accuracy. If a platform is generating images at scale (thousands per hour), the additional cost of external API calls (search, quality scoring) and multiple serial generation passes could dominate the total cost of deployment. The paper's comparison to commercial models (Nano Banana-2 at 88.5 vs. Unify-Agent at 73.2) is incomplete without knowing whether those commercial models achieve their scores with lower latency.

What evidence exists in the paper. None. The training hyperparameters in Table 6 (64 H20 GPUs, 10 days, 10K steps) provide some sense of training cost, but inference-time cost is never characterized. The paper does not report the number of flow-matching steps used for generation, the average number of candidate images evaluated during visual search, the average wall-clock time per prompt, or the total FLOPs per generation relative to baselines.

Mitigation status. Not addressed. The paper makes no mention of latency, throughput, or deployment cost in either the main text or the limitations section. This is a significant omission for a systems paper that proposes a new inference-time paradigm.

6.5 Single Base Model, Single Task Domain, No Cross-Architecture Validation

The assumption or constraint. All experiments use Bagel-14B as the base unified multimodal model and evaluate exclusively on knowledge-intensive image generation benchmarks (FactIP, WiSE, KiTTEN, T2I-FactualBench). The paper does not test whether the agentic fine-tuning approach transfers to other UMM architectures (Janus, Emu3, Show-o) or other generation domains (e.g., code generation from documentation, 3D asset creation from reference images, scientific figure generation). The paper's architectural claims — particularly the "generation improves understanding" synergy from dual VAE+ViT encoding (Section 6.4) — are validated on a single model family with a specific visual encoding design.

The consequence. A practitioner using a different unified model (e.g., Janus-Pro with its decoupled understanding and generation encoders, or Emu3.5 with its native multimodal architecture) cannot assume that Unify-Agent's agentic fine-tuning recipe will produce comparable gains. The dual VAE+ViT encoding that the paper argues is central to recaptioning quality may not have an analog in other architectures. If the gains are partially specific to Bagel's architecture, the paper's broader claims about the value of end-to-end unified agentic generation may overstate generalizability.

Furthermore, the exclusive focus on image generation leaves open whether the agentic paradigm (gap detection → evidence acquisition → specification transformation → grounded generation) applies to other modalities. The paper's motivating argument — that failures on long-tail concepts stem from knowledge gaps, not rendering quality — should apply equally to text generation (generating accurate descriptions of obscure entities), code generation (generating API calls for rarely-used libraries), or 3D generation (creating models of specific real-world objects). But without evidence in any other domain, the claimed reframing of "world-grounded generation as an inference-time sequential decision process" (Section 1) remains demonstrated only for the narrow case of image synthesis of visually-grounded IPs.

What evidence exists in the paper. The paper consistently uses "we believe this model is representative" framing without empirical support. The ablation studies (Table 5) measure component contributions within Bagel-14B only. The paper reports strong results on multiple benchmarks, but all are text-to-image benchmarks evaluating factual and identity faithfulness — there is no evidence from non-image domains or alternative model architectures.

Mitigation status. Section 7 identifies future work on "stronger unified backbones" but does not frame the single-model limitation as a current constraint. The paper's claims about architectural synergy (Section 6.4) are presented as general insights without appropriate caveats about the single-architecture validation. No experiments on alternative UMMs are proposed or conducted.

6.6 Reliance on External Commercial Systems for Both Data Construction and Evaluation

The assumption or constraint. The entire Unify-Agent training and evaluation pipeline depends on a chain of external commercial models, each introducing potential biases and reproducibility barriers. The training data is constructed using Claude Opus 4.6 (trajectory synthesis), Gemini 3 Flash (image quality filtering), Nano Banana Pro (generation-based verification), and GPT-4o (recaption quality judging). The evaluation uses Seed2.0 (FactIP) and GPT-4o (WiSE, KiTTEN, T2I-FactualBench) as judges. None of these models are open-source or guaranteed to remain available with consistent behavior.

The consequence. Reproducibility is fundamentally limited. A researcher attempting to replicate the training pipeline in the future may find that Claude Opus 4.6 has been deprecated, that GPT-4o's judging behavior has changed due to model updates, or that Seed2.0 is not publicly accessible. More subtly, the biases of these teacher and judge models are baked into the training data and evaluation scores in ways the paper does not characterize. If Claude Opus 4.6 systematically formulates search queries in a particular style, Unify-Agent learns that style. If Seed2.0 has aesthetic preferences (e.g., favoring photorealistic over stylized renderings), the FactIP scores reflect those preferences. The paper's strong results may thus be partially attributable to alignment between the teacher models used for data construction and the judge models used for evaluation — a form of "evaluator-teacher collusion" that inflates apparent performance relative to what a human evaluator or a different judge model would assign.

What evidence exists in the paper. The paper specifies which models are used for which roles (Sections 4.1 and 6.1, Appendices C and E) but provides no analysis of inter-model agreement, no human validation of judge scores, no comparison of scores under alternative judges, and no discussion of how teacher model biases might propagate into trained behavior. The FactIP evaluation protocol uses Seed2.0 as the sole evaluator; WiSE, KiTTEN, and T2I-FactualBench use GPT-4o as the sole evaluator. There is no cross-validation of Seed2.0's judgments against GPT-4o's or against human raters.

Mitigation status. Not addressed. The paper treats the choice of teacher and judge models as implementation details rather than as methodological variables. The Appendix C evaluation protocol describes the evaluation prompts in detail but does not validate the judges against ground truth or against each other. For a paper that introduces a new benchmark (FactIP) and reports state-of-the-art results across multiple existing benchmarks, the absence of evaluator reliability analysis is a meaningful gap — particularly given that the paper's contributions include the benchmark itself, where the evaluator's properties directly determine what it means for a model to "perform well."

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a conceptual reframing of world-grounded image synthesis rather than a paradigm shift. It does not claim that agentic pipelines are universally superior to closed-book generation, nor does it argue that retrieval augmentation replaces the need for large-scale pretraining. Instead, it makes a more specific and empirically grounded contribution: for long-tail, knowledge-intensive visual concepts where parametric memory is insufficient, recasting generation as an inference-time sequential decision process — with explicit gap detection, evidence acquisition, and evidence transformation — can substantially improve identity fidelity compared to closed-book generation and to naive retrieval augmentation.

The magnitude of this reframing is significant but bounded. The paper demonstrates that Unify-Agent improves over its base model Bagel by over 22 points on FactIP (50.9 → 73.2, Table 1) and lifts entity alignment on KiTTEN from 2.64 to 3.93 (Table 3). These are large, practically meaningful gains — not incremental tweaks. However, the approach does not surpass the strongest commercial systems (Nano Banana-2 achieves 88.5 on FactIP, Table 1), and it does not address hard problems where the base model's generative capability is fundamentally insufficient (the T2I-FactualBench MKCC composition score of 71.5 trails DALL-E 3's 75.7, Table 4). The reframing is thus domain-specific in its benefits: it improves performance precisely where world knowledge is the bottleneck, not where generation quality or compositional reasoning is the bottleneck.

The paper's reconciliation of a conceptual tension in the literature is one of its most landscape-shifting contributions. The unified multimodal model community has been building architectures that natively support both understanding and generation (Janus, Show-o, Emu3, Bagel), motivated by the intuition that these capabilities should mutually reinforce each other. Yet these models have remained closed-book systems, and their performance on factual benchmarks has lagged behind dedicated generation models. The paper's Bagel-CoT baseline — which adds chain-of-thought reasoning to Bagel but no tool use — actually reduces performance on FactIP (50.9 → 47.0, Table 1) and entity alignment on KiTTEN (2.64 → 2.02, Table 3). This negative result is substantively important: it demonstrates that better internal reasoning within a closed-book UMM does not automatically translate to better generation for knowledge-intensive concepts. The UMM community's implicit assumption — that reasoning capability would compensate for knowledge gaps — is shown to be false.

Unify-Agent's positive results then demonstrate the necessary condition for UMMs to fulfill their promise on factual generation: the reasoning capabilities must be coupled with external evidence acquisition and transformation. This is not an obvious finding. One might have hypothesized that a UMM with strong reasoning could "reason its way" to correct visual attributes by combining partial parametric memories, but the Bagel-CoT result strongly contradicts this. The paper thus provides both a diagnostic (closed-book reasoning is insufficient) and a remedy (agentic evidence acquisition and recaptioning), which together clarify what UMMs need to succeed on world-grounded tasks.

The paper also redirects research attention in several important ways:

Away from scaling pretraining data for long-tail coverage. The paper's motivating argument — that there will always be long-tail concepts outside any training distribution — implies that attempts to solve factual generation by simply collecting more training data are asymptotically inefficient. This strengthens the case for inference-time knowledge acquisition as a complementary strategy to pretraining scaling.
Away from decoupled retrieval-augmented pipelines. The paper's finding that naive multimodal injection is suboptimal (text + visual injection performs worse than visual alone, Figure 2) and that recaptioning provides substantial additional gains (Table 5: removing recaptioning drops Overall by 10.3 points) suggests that loosely coupled agentic systems — where an LLM retrieves text and images and passes them to a frozen generator — are architecturally limited. The evidence supports tighter integration where the generating model itself performs the evidence transformation.
Toward architectural designs that make generative features available for understanding tasks. The paper's recaption architecture ablation (Table 5: removing VAE from recaptioning drops Overall by 2.0 points, removing ViT drops it by 11.8) provides specific, quantitative evidence that generative components (VAE latents) contribute to understanding tasks (evidence interpretation). This suggests that UMM architectures should consider exposing low-level perceptual features during visual reasoning tasks, not just during generation — a design principle that current UMMs largely do not follow.

Follow-Up Research This Work Enables

Testing whether the agentic training generalizes across base UMM architectures. The paper validates Unify-Agent on Bagel-14B only, and several of its architectural claims — particularly the VAE+ViT synergy during recaptioning — are specific to Bagel's dual visual encoding design. A direct next step is to replicate the agentic fine-tuning pipeline (same 143K trajectories, same SFT procedure with sequence packing and hybrid attention masking) on other unified multimodal architectures: Janus-Pro-7B (which uses decoupled understanding and generation encoders), Emu3.5 (which uses native multimodal modeling without explicit VAE/ViT separation), and Show-o (which uses discrete visual tokens rather than continuous latents). The key measurement would be: does the recaptioning quality improvement transfer, and if so, is the mechanism the same? If Janus-Pro — which lacks VAE-level perceptual latents during understanding — shows comparable recaptioning gains, the paper's architectural synergy claim would be weakened. If gains are substantially smaller, the claim would be strengthened and would provide specific guidance for UMM architecture design.

Characterizing the dependence on search quality and identifying the failure boundary. The paper provides no analysis of how Unify-Agent behaves when external search returns low-quality or incorrect results — a common real-world scenario. A controlled experiment would systematically degrade search quality and measure generation fidelity: (1) replace top-2 retrieved images with random images of unrelated entities, (2) replace reference images with images of visually similar but incorrect entities (e.g., a different mascot with similar features), (3) provide reference images with progressively heavier watermarks or lower resolution, (4) provide text search results that contain factual errors. For each degradation type, measure FactIP Relevance scores and also the model's self-awareness — does it recognize when references are poor and either search again or refuse to generate? This would establish the robustness boundary of the approach and identify whether the quality-filtering mechanism (Section 4.1.2) actually works at inference time or is primarily a training-data construction tool.

Measuring whether recaptioning transfers to tasks without visual ground truth. The recaption module is designed to transform retrieved evidence into generation-optimized specifications. Its core capability — looking at reference images and producing structured descriptions that disentangle identity from composition — might transfer to tasks where no reference images exist but detailed text descriptions are available. For example, generating an image of "a typical 17th-century Dutch merchant's home interior" requires factual grounding (what furniture, lighting, architectural details are period-appropriate) that could be acquired through text search and structured into a recaption even without visual references. A transfer experiment would test Unify-Agent on prompts from domains like historical reconstruction, scientific illustration, or architectural visualization where factual knowledge is primarily textual. If recaptioning provides gains in these zero-reference-image settings, it would demonstrate that the capability is about evidence organization generally, not specifically about visual evidence interpretation.

Ablating the teacher model quality to understand data construction sensitivity. The paper constructs training trajectories using Claude Opus 4.6 and evaluates image quality using Gemini 3 Flash — both strong commercial models with specific capabilities and biases. A systematic ablation would construct parallel training datasets using weaker teacher models (e.g., open-source models like Llama-3-70B for trajectory construction, Qwen-VL for image quality scoring) and measure how Unify-Agent's performance degrades as a function of teacher capability. The 68% data rejection rate (456K → 143K, Section 4.1.3) suggests that trajectory quality is highly variable; the key question is whether this variability is primarily driven by the teacher model's reasoning capability (which might be substituted with a weaker model plus more aggressive filtering) or by fundamental difficulty of the IP concepts (which would require strong teachers regardless). If performance degrades smoothly with teacher capability (e.g., linear in teacher benchmark scores), then future work can use cheaper teachers with more aggressive filtering. If performance degrades sharply (a cliff), then the approach fundamentally depends on frontier-level teacher models, which has significant reproducibility implications.

Multi-round iterative refinement with self-verification. The paper explicitly identifies the single-pass limitation (Section 7) but does not implement or test iterative variants. A natural extension is a two-pass system: after the first generation, the model evaluates whether identity fidelity is sufficient (by comparing the generated image to the reference images), and if not, performs targeted additional search for the specific attributes that were captured incorrectly (e.g., "the hair color was wrong — search for images emphasizing hair detail") and regenerates. The key measurement would be: for prompts where the single-pass system produces incorrect identity features, what fraction are corrected by one additional refinement round? And at what point do diminishing returns set in (2 rounds? 3?)? This experiment would establish whether the single-pass design is a minor limitation (most errors would be fixed by one extra round) or a fundamental constraint (errors compound despite refinement). The paper's finding that 68% of initial trajectories were discarded during training construction suggests that even strong teacher models frequently produce incorrect recaptions on the first attempt; iterative refinement might substantially increase the yield of correct generations.

Human evaluation of FactIP to calibrate MLLM-as-judge reliability. The paper's FactIP evaluation uses Seed2.0 as the sole evaluator with a heavily weighted Relevance dimension (0.75 of the total score, Appendix C). A calibration study would collect human judgments (say, from 5–10 raters per sample) on a representative subset of FactIP (e.g., 200 prompts stratified by category and difficulty) and measure: (1) inter-rater agreement among humans for Relevance judgments, (2) correlation between Seed2.0's Relevance scores and mean human Relevance scores, (3) whether Seed2.0 exhibits systematic biases (e.g., over-penalizing stylized generations, under-penalizing subtle identity errors). If human-SEED2.0 correlation is high (say, Pearson r > 0.8), the benchmark's validity is strengthened; if low, the reported scores — and the 22.3-point gap between Unify-Agent and Bagel — would need to be interpreted more cautiously. This is particularly important because FactIP is introduced as a contribution of the paper; the community needs to know whether it measures what it claims to measure.

Practical Applications and Downstream Use Cases

Brand-consistent content creation for IP holders and creative agencies. The paper's strongest results are on character and mascot categories: Unify-Agent scores 75.7 on Mascot and 71.9 on Celebrity (Table 8), with particular strength in preserving identity-critical facial features, costume details, and defining visual traits. A creative agency managing a brand's mascot across marketing campaigns could use Unify-Agent (or a system trained on similar principles) to generate on-brand imagery where the mascot appears in diverse seasonal contexts, promotional scenarios, or stylistic treatments without requiring manual illustration for each variant. The key value proposition is that the system preserves identity fidelity (the mascot consistently looks like itself) while allowing compositional flexibility (different backgrounds, poses, props). The 27.5-point Relevance improvement over vanilla Bagel (44.9 → 72.4, Table 5) translates directly to fewer identity-drifting generations that would require human correction. A practical deployment would fine-tune the system on the specific brand's IPs using the same trajectory construction pipeline, potentially with a much smaller curated set than the full 143K.

Historical and educational visualization with factual accuracy constraints. Unify-Agent scores 80.2 on Food, 76.5 on Mythology, and 74.0 on Festival (Table 8) — categories where factual grounding to specific cultural referents matters for educational accuracy. A textbook publisher or educational content platform could use an agentic generation system to produce historically accurate illustrations of, for example, specific cultural artifacts, regional cuisines, or traditional celebrations, where generic T2I models would produce plausible-but-wrong depictions. The WiSE results (Table 2) show strong performance on Cultural (0.82) and Biology (0.72), suggesting that the evidence acquisition pipeline works for domain-specific factual knowledge beyond just IP characters. The practical workflow would involve curating reference image databases for the specific entities relevant to the educational content (analogous to the FactIP seed image collection), then using the agentic pipeline to generate varied compositions while preserving factual accuracy. The 0.25 WiScore improvement over Bagel (0.52 → 0.77) represents a substantial reliability gain in an application where factual errors have pedagogical consequences.

E-commerce product visualization for long-tail and custom items. The FactIP Object category results — 80.2 on Food, 76.3 on Toy, 75.5 on Animal — demonstrate Unify-Agent's ability to preserve object-specific visual identity. An e-commerce platform with millions of SKUs, many of them from small manufacturers with limited product photography, could use an agentic generation system to create lifestyle imagery showing products in context (e.g., "this specific designer toy on a desk surrounded by art supplies") while ensuring the product itself is depicted accurately. The critical capability is that the model can retrieve reference images of the specific product variant (via the image search and quality filtering pipeline), extract identity-defining features via recaptioning, and then compose those features into new scenes — exactly the disentanglement of identity and composition that the recaption module is designed for. The paper does not provide direct e-commerce evaluations, but the strong Object Relevance scores (71.8 overall, Table 1) compared to generation-only models (FLUX.1-dev scores 8.2 Relevance on Objects) suggest meaningful gains over closed-book generation for specific product depiction.

Dependency tree

Papers this one builds on, walking backwards through the same domain and shared topics.

DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Search2025· llm-systems← LLM-judged similar
- WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent2025· multimodal← LLM-judged similar
- When to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensembling2025· llm-systems← in-domain predecessor
- V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Models2025· llm-systems← similar topic
Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification2026· agents← in-domain predecessor
- CodeClash: Benchmarking Goal-Oriented Software Engineering2025· code← LLM-judged similar
- T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search2026· agents← in-domain predecessor
- MM-WebAgent: A Hierarchical Multimodal Web Agent for Webpage Generation2026· agents← similar topic
FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation2026· agents← similar topic
- Agentic Reasoning for Large Language Models2026· agents← in-domain predecessor
- Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization2025· training-methods← topic thread
- VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory2026· agents← similar topic

What it influenced

Later papers in the same chain. Direction reversed.

CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery2026· agents→ in-domain predecessor
- GPA: Learning GUI Process Automation from Demonstrations2026· agents→ in-domain predecessor
Scaling Agent Learning via Experience Synthesis2025· agents→ similar topic
- Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystem2025· agents→ LLM-judged similar
- ExpSeek: Self-Triggered Experience Seeking for Web Agents2026· agents→ LLM-judged similar
- Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text2026· agents→ LLM-judged similar
- EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience2026· agents→ LLM-judged similar
- SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue2026· agents→ LLM-judged similar
- Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning2026· agents→ LLM-judged similar
- Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models2026· data→ LLM-judged similar
- Robot Learning from a Physical World Model2025· agents→ in-domain predecessor
- Agent Learning via Early Experience2025· agents→ similar topic
- UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2025· agents→ similar topic
- Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience2025· agents→ similar topic
FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation2026· agents→ similar topic
- The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning2026· reasoning→ topic thread
- VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memory2026· agents→ similar topic
ScaleEnv: Scaling Environment Synthesis from Scratch for Generalist Interactive Tool-Use Agent Training2026· agents→ similar topic
- D-CORE: Incentivizing Task Decomposition in Large Reasoning Models for Complex Tool Use2026· training-methods→ topic thread
- SIMA 2: A Generalist Embodied Agent for Virtual Worlds2025· agents→ similar topic
- Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence2026· agents→ similar topic