ArXiv: 2604.08644

🎯 Pitch

LG’s new open-weight vision-language model outperforms Qwen’s 235B-parameter behemoth on document understanding and math-vision benchmarks, despite being 7× smaller. This 33B dense model achieves these gains by deliberately skewing its training data toward document-centric and Korean-language corpora, proving that targeted curation trumps raw scale for industrial tasks.

1. Executive Summary

This technical report introduces EXAONE 4.5, LG AI Research's first open-weight vision-language model that integrates a dedicated 1.2B-parameter vision encoder into the existing EXAONE 4.0 32B language model framework. The model is trained on large-scale multimodal data with a targeted data design emphasizing document-centric corpora and Korean cultural content, evaluated against models including GPT-5 mini, Qwen3-VL-235B, and Qwen3.5-27B on vision benchmarks (MMMU, MATHVISION, OMNIDOCBENCH) and language benchmarks (AIME 2026, LIVECODEBENCH V6). The architecture employs several named mechanisms—hybrid attention with Grouped Query Attention (GQA) in both the vision encoder and language decoder for end-to-end computational efficiency, Multi-Token Prediction (MTP) for improved decoding throughput, 2D Rotary Positional Embedding (2D RoPE) for spatial structure capture in the vision encoder, and a multi-stage pre-training curriculum progressing from foundational modality alignment to perceptual and knowledge refinement—enabling the model to outperform the substantially larger Qwen3-VL-235B on MATHVISION (75.2 vs. 74.6) and CHARXIV (71.7 vs. 66.1) while extending context length to 256K tokens via context extension embedded directly in supervised fine-tuning. The report establishes that a dense 33B-parameter model can match or exceed 235B-parameter mixture-of-experts alternatives on document understanding and Korean contextual reasoning, but only when the training data is deliberately skewed toward those specific industrial application domains rather than relying on scale alone.

2. Context and Motivation

The Core Problem: Vision-Language Models That Don't Understand Documents and Non-English Contexts

The fundamental problem EXAONE 4.5 addresses is deceptively straightforward: existing vision-language models (VLMs) perform poorly on document understanding tasks and on languages beyond English, even when they excel at general visual reasoning. The paper emerges from LG's observation that industrial deployment of VLMs requires capabilities that current state-of-the-art models systematically underdeliver on: parsing structured documents (tables, charts, blueprints, technical manuals), handling Korean-language academic and cultural content, and operating within enterprise-scale long-context scenarios (Section 1). These are not edge cases for LG—they are the core use cases in manufacturing quality control, enterprise engineering, maintenance diagnostics, and compliance checking.

This gap matters because VLMs are increasingly being positioned as general-purpose tools for industry, yet the benchmark landscape that drives model development is dominated by English-language, general-domain evaluation (MMMU, MMMU-PRO, MathVista). A model that scores well on these benchmarks may still fail when asked to read a Korean engineering diagram or cross-reference a technical manual with a visual blueprint in a 100K-token context. The paper frames this as a capability gap that scale alone does not close—a central claim supported by the finding that EXAONE 4.5 (33B dense) outperforms Qwen3-VL-235B (a 236B MoE model with approximately 7× more total parameters) on document-centric benchmarks like CHARXIV (71.7 vs. 66.1) and OMNIDOCBENCH V1.5 (81.2 is competitive, though Qwen3-VL-235B scores 84.5), purely through targeted data curation rather than parameter scaling (Table 2).

The problem has both practical and theoretical dimensions. Practically, LG—and any organization operating in non-English industrial contexts—needs VLMs that work in their actual deployment environments. Theoretically, the paper challenges the implicit assumption in the VLM scaling community that "bigger model + more general data = better everywhere." EXAONE 4.5 demonstrates that domain-specific data strategy can compensate for parameter count differences of roughly an order of magnitude, but only in the domains where that strategy is applied.

The Industrial AI Context: From LLMs to VLMs to VLAs

The paper positions EXAONE 4.5 within a deliberate architectural trajectory that LG has been pursuing across multiple model generations (Section 1). Understanding this lineage is important because it reveals why a vision-language model matters now rather than earlier, and why the architecture choices are what they are:

  • EXAONE 3.0 and 3.5 were pure LLMs focused on integrating generative AI into industrial applications. They handled text but not images.
  • EXAONE Deep was a specialized reasoning model for mathematics, science, and coding—still text-only, but pushing on logical reasoning capabilities that industrial diagnostics would later require.
  • EXAONE 4.0 introduced a dual-mode architecture with NON-REASONING and REASONING modes, allowing the same model to handle both quick general-purpose tasks and complex multi-step problems. This is the 32B language backbone that EXAONE 4.5 inherits.
  • EXAONE 4.5 adds vision to this stack, producing LG's first VLM. The paper explicitly states this is not an endpoint but a stepping stone toward Vision-Language-Action (VLA) models that can "autonomously interact with and operate within physical industrial environments" (Section 1).

This trajectory clarifies the motivation: EXAONE 4.5 is not a research project exploring whether VLMs are useful. It is an engineering milestone in a planned progression toward physical AI systems. The paper's focus on document understanding (rather than, say, natural image captioning or visual commonsense reasoning) makes sense in this context—industrial deployment of VLMs in manufacturing and maintenance involves reading blueprints, checking compliance against manuals, and parsing structured documentation, not describing scenic photographs.

Where Existing VLMs Fall Short: Three Specific Gaps

The paper identifies three categories of limitations in prior work, though it does not enumerate them as a formal taxonomy. I extract them from the architecture and data design sections:

Gap 1: Visual Token Processing Creates a Computational Bottleneck

Existing VLMs face a fundamental tension: high-resolution images produce many visual tokens, but aggressively reducing token count destroys spatial and semantic information (Section 2.1). The paper notes that "a naive reduction in the number of visual tokens significantly degrades performance by discarding critical spatial and semantic information." Prior approaches typically use relatively small vision encoders (the paper cites ~600M parameters as representative, Section 2.1), which forces the model to either truncate tokens or produce compressed, lossy representations.

For document understanding specifically, this is catastrophic. A scanned engineering diagram or a dense table requires high resolution to be legible—compressing it into a small number of visual tokens obscures the fine-grained text and structural details that document parsing depends on. EXAONE 4.5's solution is to use a 1.2B-parameter vision encoder trained from scratch, roughly 2× the scale of the typical vision encoder in comparable models, combined with hybrid attention and GQA to maintain computational efficiency despite the larger encoder.

The paper's claim that "existing vision encoders did not meet our requirements in terms of scalability and efficiency" (Section 2.1) is telling. It implies that LG evaluated off-the-shelf vision encoders (possibly SigLIP, CLIP variants, or others used in comparable VLMs) and found them inadequate for their industrial document-centric use case. Rather than adapting an existing encoder, they trained one from scratch using an autoregressive objective inspired by OpenVision2 (a recent generative pretraining approach for visual encoders), ensuring alignment with their specific requirements from the ground up.

Gap 2: Non-English Languages and Culturally Specific Content Are Systematically Underrepresented

This is the most explicitly documented gap. The paper states that to "address the under-representation of Korean cultural and linguistic nuances, we curate a specialized Korean multimodal corpus" (Section 2.2.2). The data sources named are concrete and revealing:

  • Korea Tourism Organization (KTO) datasets covering historical and contemporary Korean culture
  • IT Donga and Game Donga for Korean digital culture, gaming, and IT content
  • Rendered Korean academic content produced by converting text-based problems into high-resolution images (a text-to-vision augmentation strategy)

The problem is not just that Korean-language data is scarce in typical web-crawled training sets (which it is). It is that even when Korean text is present, the visual contexts that accompany that text—the charts, diagrams, infographics, and document layouts specific to Korean media, academia, and industry—are essentially absent. A VLM trained predominantly on English-language visual content develops visual reasoning capabilities that are implicitly tied to English-language visual conventions and cultural reference points.

The paper demonstrates this gap concretely through the KMMMU benchmark results (Table 2), where EXAONE 4.5 scores 42.7 versus Qwen3-VL-235B's 42.1—essentially tied, but this is a benchmark designed to test Korean multimodal understanding specifically. The more revealing comparison is K-VISCUIT (80.1 for EXAONE 4.5 vs. 78.5 for Qwen3-VL-235B), which evaluates visual and cultural interpretation. These are not massive margins, but they demonstrate that targeted Korean data curation can close the gap with much larger models that have seen vastly more total data but less Korean-specific content.

The paper's approach to the Korean gap is notable for its grounding strategy: to minimize hallucinations, "generation is grounded in factual information, ensuring responses regarding Korea are based on verifiable evidence" (Section 2.2.2). This is a response to a known failure mode of LLMs and VLMs—generating plausible-sounding but incorrect information about cultures and topics where the training data is sparse. By using authoritative sources (KTO, established Korean publications) rather than indiscriminate web crawling, the model's Korean knowledge is qualitatively different from what a general-domain model would acquire.

Gap 3: Document Understanding Requires Structured Data That General Image-Text Pairs Do Not Provide

The paper's pre-training data breakdown (Section 2.2.2) reveals that general-domain image caption data—the backbone of most VLM training—is inadequate for document understanding. The problem is that image captions describe what is in the image at a semantic level ("a person sitting at a desk with a computer"), while document understanding requires structural parsing of the image's content—recognizing that this region of pixels is a table with specific rows and columns, that this text block is a header with a particular hierarchical relationship to the following text, that this chart axis represents a specific variable.

EXAONE 4.5 addresses this through multiple specialized data pipelines:

  • OCR data at the character, word, and document levels, including synthetic OCR images with "diverse backgrounds and contrastive pairs of visually confusable words" (Section 2.2.2). The contrastive pairs are particularly important—they force the model to learn fine-grained visual discrimination between similar-looking characters, which is exactly what OCR demands.
  • Document parsing tasks that transform charts, tables, and documents into structured formats (HTML, Markdown, JSON), teaching the model to extract layout and semantic structure rather than just describe content.
  • Grounding data with bounding box annotations normalized to [0,1000] coordinates, following established conventions (the paper cites Qwen3-VL's approach). This teaches spatial localization—knowing where in the image an object or text region is, not just what it is.
  • STEM and reasoning data generated through a search-based synthesis pipeline that retrieves and synthesizes domain-specific documents, addressing the scarcity of high-level academic visual content (complex mathematical graphs, engineering diagrams, scientific figures).

The contrast with standard VLM training is stark. A model trained primarily on Conceptual Captions, COCO, or similar datasets learns to associate images with natural language descriptions. EXAONE 4.5 is trained to read, parse, locate, and structurally reconstruct visual documents. These are fundamentally different capabilities, and the paper's results suggest that general VLM training does not accidentally produce strong document understanding—it has to be explicitly built in.

How the Paper Positions Itself: Not a New Architecture, But a New Data Strategy

The paper does not claim architectural novelty. Section 2.1 explicitly acknowledges that hybrid attention, GQA, 2D RoPE, and MTP are all established techniques from prior work (Ainslie et al., 2023; Su et al., 2023; Gloeckle et al., 2024; DeepSeek-AI, 2025). The vision encoder is trained from scratch, but using an objective from OpenVision2 (Liu et al., 2025). The language backbone is EXAONE 4.0, an existing model. The tokenizer is reused from K-EXAONE.

The paper's contribution is in how these components are assembled, trained, and—most critically—what data they are trained on. The positioning is that a well-engineered combination of known techniques, when paired with domain-targeted data curation, can produce a model that matches or exceeds much larger general-domain models on the specific tasks that matter for industrial deployment. This is a pragmatic engineering philosophy rather than a research breakthrough: the goal is not to advance the state of the art in VLM architecture but to build a model that actually works for LG's use cases and to release it openly to accelerate community research.

The paper also positions itself within the broader trend toward open-weight models. The repeated emphasis on EXAONE 4.5 being LG's "first open-weight VLM" (Section 1, Abstract, Conclusion) signals a strategic shift from proprietary deployment to community engagement. The model is released under the EXAONE AI Model License Agreement 1.2 - NC (Appendix B), which permits research and educational use but prohibits commercial exploitation without a separate agreement. This positions EXAONE 4.5 as a research artifact that also serves as a demonstration of LG's industrial AI capabilities—a dual-purpose release that advances the field while showcasing LG's technical competence to potential commercial partners.

The Unstated Motivation: Competition with Qwen and GPT in the Korean/Asian Market

While the paper does not state this explicitly, the benchmark comparisons reveal a clear competitive positioning. The primary baselines are:

  • Qwen3-VL-32B/235B: The leading open-weight VLM series from Alibaba, which has been aggressively expanding its multilingual capabilities and represents the strongest competition in Asian-language VLM performance.
  • GPT-5 mini: OpenAI's closed-weight offering, representing the frontier of English-centric VLM capability.
  • Qwen3.5-27B: A newer reasoning-focused model that sets the upper bound on pure language reasoning tasks.

EXAONE 4.5 is designed to compete on a specific axis: be the best at Korean document understanding and industrial visual tasks, while remaining broadly competitive on English-language benchmarks. The results bear this out: it beats Qwen3-VL-235B on MATHVISION (75.2 vs. 74.6), CHARXIV (71.7 vs. 66.1), and is competitive on general benchmarks like MMSTAR (74.9 vs. 78.7) and HALLUSIONBENCH (63.7 vs. 66.7). It does not need to win everywhere—it needs to win in the specific domains where LG will actually deploy it.

This competitive reality—the existence of strong, well-funded open-weight VLMs from competitors—is the unstated context that makes EXAONE 4.5's data-centric strategy necessary. LG cannot out-scale Qwen or GPT; the 33B parameter count is fixed by practical constraints (the EXAONE 4.0 base model size). The only available lever is training data composition and training strategy, and the paper is essentially a demonstration that this lever, when pulled aggressively toward a specific application domain, can compensate for a 7× deficit in total parameters.

What Makes This Problem Non-Trivial

The paper's approach might sound straightforward—"just train on more document data and Korean data"—but the implementation reveals several non-obvious challenges that prior work either ignored or handled inadequately:

1. Preventing language capability degradation during multimodal training. When you add vision to a language model, the text-only performance can degrade because the model's capacity is now partially allocated to visual processing. The paper addresses this by "integrating text-only data, including pre- and post-training datasets from the K-EXAONE pipeline" into the multimodal training objective (Section 2.2.1, Stage 1). This is a standard technique (mixing text-only and multimodal batches), but the paper's emphasis on it suggests that maintaining the strong language performance of EXAONE 4.0 was a non-trivial engineering constraint.

2. Balancing general visual capability with specialized document understanding. A model trained too heavily on documents and OCR might lose general visual reasoning. The two-stage pre-training curriculum (Section 2.2.1) is designed to address this: Stage 1 establishes broad visual-text alignment using general-domain image-text pairs, interleaved documents, and OCR samples, while Stage 2 refines toward high-density structured information by reducing general domain data and upsampling grounding, document, and STEM data. This progressive specialization prevents the model from becoming a document-only specialist that cannot handle natural images.

3. Synthetic data quality control for underrepresented domains. For Korean content and STEM diagrams, the paper relies heavily on synthetic data generation. The risk is hallucination propagation—if the synthetic generator produces incorrect content, the model learns those errors. The paper's mitigation strategies include grounding synthetic captions in existing metadata ("ensuring that descriptions are detailed and grounded in verifiable evidence"), using a text-to-vision augmentation strategy where text-based problems are converted to rendered images (retaining the original text's correctness), and balancing synthetic counting data across count ranges and object types to counteract biases. These are non-trivial data engineering decisions that distinguish careful synthetic data curation from indiscriminate generation.

4. Context extension without catastrophic forgetting. Extending context length to 256K tokens is challenging for language models alone; doing it for a VLM where the visual representations were trained at a specific context length (8K during pre-training) introduces additional risk of cross-modal alignment degradation. The paper's approach of embedding context extension directly in the SFT stage (Section 2.3), rather than as a separate phase, is motivated by the "stabilization provided by the 128K-capable base LLM"—the language backbone already handles long contexts, and the SFT data simultaneously refines instruction-following and long-range coherence while using context parallelism to handle the 256K computational requirements. This integrated approach avoids the common failure mode where a separate context extension phase disrupts previously learned capabilities.

3. Technical Approach

3.1 Reader Orientation

This paper describes the construction and training of a 33-billion-parameter vision-language model that combines a language backbone—the existing EXAONE 4.0 32B model—with a newly trained 1.2-billion-parameter vision encoder, enabling it to process both text and image inputs and produce text outputs. The system solves the problem of building a VLM that excels at document understanding and Korean-language industrial tasks without using more parameters than competitors; the shape of the solution is domain-targeted data curation applied through a multi-stage training pipeline, where the architecture itself reuses known components (GQA, 2D RoPE, MTP) but the training data composition and stage ordering are what produce the specialized capabilities.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components connected in a feedforward processing pipeline:

  1. Vision Encoder (1.2B parameters, trained from scratch): Takes an image as input and produces a sequence of visual feature vectors. Uses 2D Rotary Positional Embedding (2D RoPE) to encode spatial position and Grouped Query Attention (GQA) for computational efficiency. This is the component that transforms raw pixels into representations the language model can consume.

  2. MLP Projector (size unspecified, but described as a "merger"): A bridge module that maps the vision encoder's output representations into the same vector space as the language model's token embeddings. This is a standard connector in VLM architectures—it translates between modalities.

  3. Language Decoder (32B parameters, from EXAONE 4.0): The autoregressive text generation backbone. Uses hybrid attention and GQA. This component receives both text tokens (from the tokenizer) and projected visual tokens (from the MLP projector) as a combined input sequence and generates output text token by token.

  4. Multi-Token Prediction (MTP) Module (from K-EXAONE): An auxiliary prediction head that forecasts multiple future tokens simultaneously during training, improving decoding throughput at inference by enabling speculative decoding-style acceleration. Disabled at inference time for benchmark evaluation.

  5. Tokenizer (from K-EXAONE): Converts raw text into token IDs that the language decoder can process. Enhanced over the EXAONE 4.0 tokenizer specifically for multilingual support and Korean language processing.

Information flows as follows: an image enters the vision encoder → 2D RoPE encodes spatial structure → GQA layers process the image into visual tokens → the MLP projector maps these into the language model's embedding space → the tokenizer simultaneously converts any text prompt into text tokens → the language decoder receives the concatenated sequence of text tokens and projected visual tokens → the decoder autoregressively generates output text → (during training only) the MTP module predicts additional future tokens for training efficiency.

3.3 Roadmap for the Deep Dive

  • First, the model configuration (§3.4.1): how the vision encoder, language decoder, positional encodings, and attention mechanisms are sized and connected, because these architectural choices determine the computational budget and constrain what the training pipeline must achieve.
  • Second, the pre-training pipeline (§3.4.2): the two-stage curriculum, the data categories, and the specific composition of training tokens at each stage, because this is the core contribution—how data strategy substitutes for model scale.
  • Third, the context length extension mechanism (§3.4.3): how the model reaches 256K tokens from an 8K pre-training context, because this enables the enterprise-scale document processing that the industrial use cases demand.
  • Fourth, the supervised fine-tuning phase (§3.4.4): the multi-domain data construction and multi-stage curriculum that teach the model to follow instructions across modalities and languages.
  • Fifth, the preference optimization and reinforcement learning phases (§3.4.5): the specific algorithms (LDPO, LGROUPER, GRPO with IcePop) and their stage-specific application, because these refine the model's behavior beyond what SFT alone can achieve.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and data engineering paper whose core idea is that a carefully composed multi-stage training pipeline, with data deliberately skewed toward document understanding, OCR, and Korean content, can produce a dense 33B-parameter VLM that matches or exceeds 235B-parameter mixture-of-experts alternatives on the specific industrial tasks that matter, without requiring architectural novelty.

3.4.1 Model Configuration: How the Architecture Is Sized and Connected

The architecture is not proposed as novel; it assembles known components into a configuration optimized for high-resolution document processing at industrial scale. The key design tension is between visual fidelity (more visual tokens preserve more detail) and computational efficiency (more tokens increase the cost of the language decoder's attention operations). The paper resolves this tension through a specific combination of encoder scale, attention mechanism choice, and resolution policy.

Vision encoder scale: 1.2B parameters, trained from scratch.

The paper explicitly departs from the common practice of using relatively small vision encoders (the paper cites "e.g., 600M parameters" as representative of prior approaches, Section 2.1). The argument is causal: a larger vision encoder can "retain rich visual representations without aggressive token truncation," meaning that even though the encoder produces more tokens per image, each token carries more information, so the total number of tokens needed to represent an image at acceptable fidelity can be lower than a smaller encoder producing compressed, lossy tokens.

Training from scratch (rather than using a pre-trained vision encoder like SigLIP or CLIP) is justified by the claim that "existing vision encoders did not meet our requirements in terms of scalability and efficiency" (Section 2.1). The training uses an "autoregressive objective inspired by OpenVision2," which is a generative pretraining approach for visual encoders that treats image patches as a sequence to be predicted, similar to how language models predict text tokens. This choice ensures that the vision encoder's representations are aligned with the autoregressive nature of the language decoder from the start, rather than requiring post-hoc adaptation of a discriminatively trained vision encoder.

Grouped Query Attention (GQA) in both vision encoder and language decoder.

GQA is an attention mechanism variant that reduces the number of key and value heads relative to the number of query heads. In standard multi-head attention, each attention head has its own query, key, and value projections. In GQA, multiple query heads share a single key-value head. The computational benefit comes from reduced memory bandwidth for key-value caching during autoregressive decoding, but the paper notes an additional benefit: "Even in the absence of KV caching, GQA provides improved end-to-end computational efficiency due to reduced attention complexity and better hardware utilization" (Section 2.1). This is because GQA reduces the total number of attention matrix multiplications, which are the dominant cost in transformer layers.

The application of GQA to the vision encoder—which does not use autoregressive generation and therefore does not need KV caching—is an interesting engineering choice that the paper justifies on hardware efficiency grounds. The paper also notes that GQA is "widely supported and continuously optimized in modern inference frameworks," making it a pragmatic choice for deployment rather than a purely algorithmic one.

2D Rotary Positional Embedding (2D RoPE) in the vision encoder, 1D RoPE in the language decoder.

Standard Rotary Positional Embedding (RoPE) encodes position by rotating token embeddings in a high-dimensional space, where the rotation angle depends on the token's position in a one-dimensional sequence. This works well for text, where position is purely sequential (position 5 comes after position 4, which comes after position 3, etc.).

2D RoPE extends this to two dimensions by encoding both horizontal and vertical position. An image patch at grid position (row=3, column=7) receives a rotation that encodes both its row and column coordinates. The paper states this is because images have a "two-dimensional spatial structure" that "differs fundamentally from the one-dimensional sequential nature of text" (Section 2.1). Without 2D positional encoding, the vision encoder would have to learn spatial relationships purely from data patterns—knowing that the patch above the current patch is semantically related requires either explicit position encoding or implicit learning from many training examples. 2D RoPE provides this information directly.

The language decoder retains 1D RoPE (the standard approach) to "preserve compatibility with pre-trained textual positional representations." This is a practical choice: the EXAONE 4.0 language backbone was trained with 1D RoPE, and replacing it would break the pre-trained representations.

Multi-Token Prediction (MTP) module.

The MTP module is an auxiliary training objective that predicts multiple future tokens simultaneously, rather than just the next single token as in standard autoregressive language modeling. The paper imports this from K-EXAONE (Choi et al., 2026) and cites DeepSeek-AI (2025) and Gloeckle et al. (2024) as the originating work.

The mechanism works by adding additional prediction heads at the final layer of the language decoder, each predicting one additional future token beyond the standard next-token prediction. During training, the model is optimized to predict tokens $t+1, t+2, \ldots, t+k$ simultaneously given tokens $1 \ldots t$. During inference, MTP is disabled—the model generates one token at a time as usual. The benefit is improved decoding throughput because the MTP-trained model produces representations that are more forward-looking, enabling speculative decoding techniques where the model can propose multiple future tokens that are then verified by a separate mechanism.

The paper explicitly states that MTP is "disabled" at inference time for benchmark evaluation (Section 3.1), meaning the reported benchmark scores reflect the standard autoregressive generation without any speculative decoding acceleration.

Image resolution policy: maximum resolution aligned with real-world inputs.

Rather than supporting ultra-high-resolution images (which would increase the visual token count and thus the computational cost of every forward pass), the paper "carefully calibrate[s] the maximum image resolution during training" to match "commonly encountered real-world inputs" (Section 2.1). The specific maximum resolution values are shown in Figure 1: the architecture diagram indicates support for images up to Width 800, Height 1000, and also shows smaller configurations (Width 480, Height 500). The diagram also labels this as "Native Resolution" processing, suggesting the model processes images at their actual aspect ratios rather than forcing a square crop.

This is a practical efficiency decision with trade-offs. Limiting maximum resolution means some very high-resolution documents (e.g., a densely detailed engineering blueprint that requires extreme zoom) may lose detail. However, the computational savings are substantial because the number of visual tokens scales quadratically with resolution (or linearly in each dimension for patch-based encoders). For LG's target industrial applications—where documents are typically standardized formats rather than arbitrarily high-resolution scans—the paper's claim is that this resolution cap does not sacrifice "practical usability."

Tokenizer reuse from K-EXAONE.

The tokenizer is inherited from K-EXAONE, which is described as having "significant enhancements over the EXAONE 4.0 tokenizer, particularly regarding multilingual support and Korean language processing" (Section 2.1). The paper does not provide specific tokenizer details (vocabulary size, tokenization algorithm, byte-level encoding strategy), but the emphasis on Korean language processing is consistent with the paper's broader focus on Korean-language industrial applications. A tokenizer optimized for Korean would typically include more Korean-specific subword units, better handling of Hangul character composition, and coverage of Korean-specific punctuation and formatting conventions.

3.4.2 Pre-Training Pipeline: The Two-Stage Curriculum and Data Composition

The pre-training pipeline is the paper's central technical contribution. It consists of two stages with different data mixtures, different training objectives, and different computational budgets, designed to progress from broad visual-text alignment to specialized document and STEM understanding.

Overall structure: from general to specific.

The pipeline trains three components simultaneously: the vision encoder (from scratch), the MLP projector, and the language decoder (from EXAONE 4.0 initialization). The training objective is multimodal—the model processes both text-only and image-text sequences in the same training batches, with language modeling loss applied to text tokens regardless of whether the input included images.

The two stages are quantified in Table 1:

StageTraining ModulesImage TokensText TokensSequence LengthFLOPs
Stage 1All420B400B8K1.57×10231.57 \times 10^{23}
Stage 2All225B110B8K6.43×10226.43 \times 10^{22}

Several patterns are immediately visible. First, the total token count decreases from Stage 1 (820B total tokens) to Stage 2 (335B total tokens), but the ratio of image tokens to text tokens shifts—Stage 1 is roughly balanced (420B image, 400B text), while Stage 2 is roughly 2:1 in favor of image tokens (225B image, 110B text). This reflects the curriculum's shift toward visual specialization. Second, the computational budget (FLOPs) drops substantially in Stage 2, which likely reflects the reduced token count rather than a change in model size.

Stage 1: Foundational Modality Alignment.

Stage 1 trains all three components (vision encoder, projector, language decoder) end-to-end on a data mixture designed to establish basic visual-text alignment. The paper describes four data categories:

General domain image-text pairs: These are standard image-caption data, primarily Korean-English bilingual. The paper acknowledges a quality problem: "the brevity and noise inherent in raw web-curated captions." The solution is a "synthetic captioning pipeline" that enhances semantic richness—essentially, using an existing captioning model to generate more detailed, structured descriptions of images, rather than relying on whatever short, noisy captions accompanied the images on the web.

The pipeline is described with specific quality criteria: it "prioritizes entity diversity, visual complexity, and fine-grained details while maintaining specificity and factual accuracy" (Section 2.2.2). The factual accuracy constraint is enforced by using "existing metadata as a reference for the synthetic generator, ensuring that descriptions are detailed and grounded in verifiable evidence." This means the synthetic captioner is not freely hallucinating—it is constrained by whatever metadata is available (e.g., alt text, surrounding webpage text, file names) to produce descriptions that are accurate even when detailed.

Interleaved image-text data: These are web documents where images and text naturally alternate—think of a Wikipedia article with embedded figures, a news article with photos, or a tutorial with screenshots. The paper curates these from "open-source and in-house resources," specifically citing Mint-1t (Awadalla et al., 2024) and Omnicorpus (Li et al., 2024) as open-source sources.

The filtering mechanism is a "lightweight text-based classifier" that scores documents on "educational quality scores and STEM-related relevance." Low-value web content (ads, navigation elements, boilerplate) is filtered out, and high-information-density documents are strategically upsampled. The paper argues that preserving "the natural sequence of text and images" teaches the model to "process multimodal information within extended contexts and associate non-adjacent visual and textual cues"—essentially, learning that an image on page 3 of a document might be referenced by text on page 7.

OCR and document understanding datasets: This is the category most directly tied to the paper's industrial focus. The data is constructed at three levels: character-level (recognizing individual characters in various fonts and contexts), word-level (recognizing words, including contrastive pairs of visually confusable words), and document-level (parsing full documents into structured formats).

The contrastive pairs are a specifically interesting design choice. By generating synthetic OCR images where the model must distinguish between visually similar words (e.g., "clay" vs. "day" in certain fonts, or Korean characters that differ by a single stroke), the training forces the model to attend to fine visual details that general image-text training would not emphasize.

The document parsing tasks transform "charts, tables, and documents into structured formats such as HTML, Markdown, and JSON" (Section 2.2.2). This is not just OCR—it teaches the model to understand document structure: which text belongs to which column, how table cells relate to headers, what the hierarchical organization of sections is. For industrial applications involving technical manuals and compliance documents, this structural understanding is what enables the model to answer questions like "What is the torque specification in section 4.3, table 2, row 7?"—it needs to know where section 4.3 is, which table is table 2, and which row is row 7.

OCR-centric samples: Additional OCR data specifically curated for document text recognition, distinct from the document structure parsing tasks.

Additionally, Stage 1 includes "text-only data, including pre- and post-training datasets from the K-EXAONE pipeline" (Section 2.2.1). This is the anti-catastrophic-forgetting measure: without text-only data, the model might gradually lose its language capabilities as it allocates capacity to visual processing. By keeping text-only data in the training mix, the model maintains its ability to handle pure text tasks (summarization, translation, code generation) even as it acquires visual capabilities.

Stage 2: Perceptual and Knowledge Refinement.

Stage 2 shifts the data mixture toward "high-density, structured information" by reducing the proportion of general domain data and upsampling three specialized categories:

Grounding and counting data: Grounding refers to the ability to locate objects in images by producing bounding box coordinates. The paper uses a standard format: "bounding boxes [x1, y1, x2, y2] denoting the top-left and bottom-right corners," with each coordinate "normalized by the image width or height and scaled to a range of [0, 1000]" (Section 2.2.2). This normalization to 0-1000 follows the convention established by Qwen3-VL (Bai et al., 2025) and means the model expresses coordinates as integers in this range, which are then divided by 1000 during processing to obtain normalized coordinates.

The grounding data comes from both "high-quality open-source sets" and an "in-house synthetic pipeline." The synthetic pipeline is necessary because real-world grounding annotations are expensive to produce (humans must draw boxes around objects) and often have inconsistent quality.

For counting data, the paper identifies a specific problem with real-world data: "noise (e.g., occlusion, crowding)" makes counting difficult even for humans, so training a model on noisy counting ground truth would teach it wrong answers. The solution is to prioritize synthetic generation where the ground-truth count is known exactly. The paper also notes a second problem: real-world counting datasets are biased toward "low count ranges and simple categories" (most images contain 1-5 objects of common types). The mitigation is "explicit balancing across count ranges and object types, followed by iterative refinement for increased difficulty and diversity."

STEM and reasoning data: This is generated through a "search-based synthesis pipeline" that addresses the scarcity of high-level academic visual content. The pipeline works by retrieving domain-specific documents (mathematical graphs, engineering diagrams, scientific figures) from some source (presumably the web or an academic database), then synthesizing question-answer pairs about them. The paper notes that Long Chain-of-Thought (CoT) data is generated for these examples, which means the training data includes not just the answer but a step-by-step reasoning trace that shows the model how to derive the answer from the visual input.

The curriculum design here is described as "progressive": "an initial broad filtering policy ensures general visual diversity, followed by the strategic upsampling of these specialized datasets to target remaining performance gaps." This suggests that the team monitored performance on held-out evaluation sets during training and adjusted the data mixture to address specific weaknesses—a data-centric version of active learning.

Korean-specific data: This is perhaps the most distinctive data category. The sources are concrete and named:

  • Korea Tourism Organization (KTO) datasets: Images and descriptions of Korean historical and contemporary culture. The paper states this "facilitates the model's acquisition of profound cultural knowledge"—learning, for example, what specific Korean architectural styles look like, what traditional Korean dishes are called and what they look like, and how Korean cultural practices are visually represented.
  • IT Donga and Game Donga datasets: Korean IT and gaming content. The paper notes these are "prevalent among active users," meaning the model is being trained on the kind of content that Korean users actually consume, rather than translated English content.
  • Rendered Korean academic content: This is a "text-to-vision augmentation" strategy where text-based problems (presumably from Korean exams, textbooks, or academic papers) are converted into "high-resolution rendered images." The purpose is to teach the model to parse and solve "formatted Korean academic content" in its visual form, which is how such content would appear in a scanned document or a screenshot.

The Korean data also has an explicit anti-hallucination mechanism: "generation is grounded in factual information, ensuring responses regarding Korea are based on verifiable evidence" (Section 2.2.2). This means the synthetic captioning pipeline for Korean content uses authoritative sources as ground truth rather than generating plausible-sounding but unverified descriptions.

Design choice: why two stages rather than one continuous training run?

The paper does not explicitly justify the two-stage design, but the logic is implicit in the data composition shift. Training on specialized document, grounding, and Korean data from the beginning could prevent the model from ever learning general visual-text alignment—it would become a document specialist that cannot handle natural images. Conversely, training only on general data and never specializing would produce a model that performs well on general VLM benchmarks but fails on LG's actual industrial tasks.

The two-stage approach is a form of curriculum learning: establish a broad foundation first (Stage 1), then specialize without losing that foundation (Stage 2). This is analogous to the common LLM training strategy of pre-training on broad internet data followed by domain-specific fine-tuning, but executed within the pre-training phase itself rather than as a separate post-training step.

3.4.3 Context Length Extension: Reaching 256K Tokens

The model supports a maximum context length of 256K tokens, which is a substantial increase from the 8K sequence length used during pre-training (Table 1). The mechanism for achieving this is architecturally simple but training-strategy novel: context extension is embedded directly in the supervised fine-tuning (SFT) stage rather than being a separate training phase.

Why this is non-standard. In typical LLM training pipelines, context length extension is a standalone phase that occurs between pre-training and SFT. The model is first pre-trained at a moderate context length (e.g., 8K), then goes through a "context extension" phase where it is trained on longer sequences (e.g., 32K, 128K, or beyond) using a modified positional encoding or attention mechanism, and finally undergoes SFT on instruction-following data. The paper explicitly contrasts with this: "Unlike conventional pipelines that treat context expansion as a standalone phase, this approach leverages high-quality fine-tuning data to simultaneously refine instruction-following capabilities and long-range coherence" (Section 2.3).

The rationale: leveraging the base LLM's existing long-context capability. The paper notes that the EXAONE 4.0 language backbone was already "128K-capable," meaning it had been trained or fine-tuned to handle contexts up to 128K tokens in its text-only form. This provides a "robust long-context prior" that "minimizes optimization instability and prevents performance degradation commonly observed when scaling from shorter sequences" (Section 2.3). In other words, the language decoder already knows how to attend over long text sequences; the challenge is ensuring that the visual representations remain aligned with the text at these extended lengths.

The paper also claims that "the vision encoder, already well aligned with the LLM through multimodal pretraining, contributes to stable extension in the multimodal setting" (Section 2.3). This suggests that the alignment between visual and textual representations learned during the 8K pre-training generalizes to longer sequences without requiring explicit long-context multimodal pre-training.

Computational mechanism: context parallelism. To handle the memory requirements of 256K-token sequences—where the attention matrices have dimensions proportional to sequence length squared—the paper employs "Context Parallelism" (citing Hao Liu, Matei Zaharia, and Pieter Abbeel, 2023, "Ring Attention with Blockwise Transformers for Near-Infinite Context"). Context parallelism distributes the long sequence across multiple GPUs, with each GPU computing attention for a segment of the sequence and communicating boundary information to neighboring GPUs. This avoids the quadratic memory scaling that would make 256K sequences infeasible on a single GPU.

The paper does not provide specific details on the number of GPUs used, the sequence partitioning strategy, or the communication overhead, but the reference to ring attention suggests a specific implementation: the sequence is divided into blocks distributed across devices in a logical ring, and each device computes attention for its block while passing key-value states to the next device in the ring.

Why this matters for industrial use cases. The 256K context length enables the model to process very long documents—a multi-hundred-page technical manual, a complete maintenance log, or a lengthy regulatory compliance document—in a single forward pass without chunking. For LG's target applications in manufacturing quality control and enterprise engineering, this means the model can cross-reference information from different sections of a long document (e.g., "Compare the safety procedure in section 2 with the one in appendix C and identify any contradictions") without the information loss that occurs when documents are chunked into smaller segments.

3.4.4 Supervised Fine-Tuning: Building Instruction-Following Across Modalities

The SFT phase transforms the pre-trained model (which knows how to process images and text but has not been trained to follow instructions) into an instruction-following VLM. The paper describes several design choices that distinguish this SFT phase from simpler approaches.

Data organization by domain, not unified. Rather than using a single data construction pipeline that treats all training examples uniformly, the paper "organize[s] the data by domain and apply[ies] tailored curation strategies for each" (Section 2.4.1). The domains include visual understanding, language use, reasoning, and instruction following. This per-domain approach means that an OCR training example might be constructed differently (with different prompt formats, different expected output structures, different quality filters) than a general visual question-answering example. The paper argues this is necessary because "reducing noisy supervision and improving alignment between inputs, instructions, and responses" requires different strategies for different types of tasks.

Joint training on text-only and vision-language data. The SFT phase trains on both modalities simultaneously, following the same strategy as pre-training. The paper specifically mentions that both NON-REASONING and REASONING supervision are integrated, reflecting the dual-mode design inherited from EXAONE 4.0. NON-REASONING examples are presumably direct question-answer pairs where the model should provide a concise response without showing its work. REASONING examples include chain-of-thought traces where the model explicitly walks through its reasoning steps before giving a final answer.

Multi-stage training curriculum within SFT. The SFT phase itself uses a "multi-stage training curriculum with stage-wise data organization" described as a "coordinated training strategy" (Section 2.4.1). The details of this curriculum are not specified (how many stages, what data goes in each stage, how the transition between stages is managed), but the goal is stated as enabling the model to "progressively strengthen its overall capabilities while maintaining broad coverage and balanced development throughout training."

Multilingual coverage. The SFT data covers all six supported languages: Korean, English, Spanish, German, Japanese, and Vietnamese. This multilingual instruction following is the mechanism by which the model learns to respond appropriately to prompts in each language—the pre-training data provided passive exposure to these languages, but SFT actively teaches the model to generate appropriate responses in each one.

3.4.5 Preference Optimization and Reinforcement Learning: Refining Behavior Beyond Imitation

The post-SFT phase uses two complementary techniques to refine the model's behavior: offline preference optimization (which teaches the model to prefer better responses over worse ones using pre-collected preference data) and online reinforcement learning (which lets the model generate its own responses and receive rewards, learning from its own exploration).

Offline Preference Optimization: LDPO and LGROUPER.

The paper applies preference optimization in a "multi-stage framework, where each phase is tailored to a specific objective and integrated at different points in the training pipeline" (Section 2.4.2). The objectives include OCR, chart understanding, visual recognition, dialogue, instruction following, and safety. The multi-stage aspect means that preference optimization is not applied once as a final step but multiple times, each time targeting a different capability.

The paper uses two different preference optimization algorithms depending on the task:

LDPO (Length-normalized Direct Preference Optimization) is used for vision tasks. The equation is:

LDPO(θ)=−E(x,y+,y−)∼D[log⁡σ(β(log⁡πθ(y+∣x)πref(y+∣x)−log⁡πθ(y−∣x)πref(y−∣x)))]LDPO(\theta) = -\mathbb{E}_{(x, y^+, y^-) \sim \mathcal{D}} \left[ \log \sigma \left( \beta \left( \log \frac{\pi_\theta(y^+ | x)}{\pi_{\text{ref}}(y^+ | x)} - \log \frac{\pi_\theta(y^- | x)}{\pi_{\text{ref}}(y^- | x)} \right) \right) \right]

where $\theta$ is the model being optimized, $\pi_\theta$ is the policy (the model's output distribution), $\pi_{\text{ref}}$ is a frozen reference model, $x$ is the input, $y^+$ is the preferred (better) response, $y^-$ is the dispreferred (worse) response, $\mathcal{D}$ is the preference dataset of $(x, y^+, y^-)$ triples, $\beta$ is a temperature parameter controlling how strongly the model is pushed toward preferences (set to 0.1), and $\sigma$ is the logistic sigmoid function.

What it computes: The difference between the log-probability ratio of the preferred response and the log-probability ratio of the dispreferred response, each computed relative to the reference model. The sigmoid converts this difference into a probability between 0 and 1, and taking the negative log turns it into a loss that is minimized. The expectation means this is averaged over the preference dataset.

In operational terms: the model is penalized if it assigns high probability to the "worse" response $y^-$ or low probability to the "better" response $y^+$, with the reference model $\pi_{\text{ref}}$ serving as an anchor that prevents the model from deviating too far from its initial distribution. The $\beta$ parameter (0.1) controls the strength of this penalty—lower $\beta$ means gentler optimization that is less likely to cause overfitting to the preference data.

Why this form: DPO is preferred over RLHF (which requires training a separate reward model and running online policy optimization) because it is simpler and more stable. The paper notes that LDPO "provides stable optimization through a reference model" (Section 2.4.2), which is a direct advantage of DPO over alternatives. The length-normalized variant (LDPO) adjusts for the fact that longer responses tend to have lower per-token probabilities; without length normalization, the model might learn to prefer shorter responses to maximize the probability ratio, rather than learning to prefer genuinely better content.

LGROUPER (Length-normalized GROUPER) is used for text tasks. GROUPER (Group-wise Preference Optimization with Relative Rewards) is a variant designed for datasets that contain multiple rejected responses per prompt. The equation is more complex:

LGROUPER(θ)=−E(x,y1,…,yG)∼D[1G∑i=1G(Ai⋅exp⁡(1∣yi∣log⁡πθ(yi∣x)))]LGROUPER(\theta) = -\mathbb{E}_{(x, y_1, \ldots, y_G) \sim \mathcal{D}} \left[ \frac{1}{G} \sum_{i=1}^{G} \left( A_i \cdot \exp \left( \frac{1}{|y_i|} \log \pi_\theta(y_i | x) \right) \right) \right]

where the auxiliary definitions are:

zi=ri−mean({rj}j=1G)std({rj}j=1G)z_i = \frac{r_i - \text{mean}(\{r_j\}_{j=1}^{G})}{\text{std}(\{r_j\}_{j=1}^{G})}

Ai=2⋅zi−min⁡({zj}j=1G)max⁡({zj}j=1G)−min⁡({zj}j=1G)−1A_i = 2 \cdot \frac{z_i - \min(\{z_j\}_{j=1}^{G})}{\max(\{z_j\}_{j=1}^{G}) - \min(\{z_j\}_{j=1}^{G})} - 1

Here, $G$ is the number of responses in each group (set to 4 for GROUPER), $y_i$ is the $i$-th response, $|y_i|$ is its length in tokens, $r_i$ is some scalar reward or score assigned to the $i$-th response, $z_i$ is the z-score normalized reward (number of standard deviations from the group mean), and $A_i$ is a rescaled advantage ranging from -1 to +1.

What it computes: For each group of $G$ responses (one preferred, the rest rejected), the model's log-probability for each response is computed and divided by response length (length normalization). This normalized log-probability is exponentiated (turning it into a positive score) and multiplied by the advantage $A_i$ for that response. Responses with positive advantage (better than average in the group) get positive weight; responses with negative advantage get negative weight. The model is trained to increase probability for positively-advantaged responses and decrease probability for negatively-advantaged ones.

The advantage normalization is a key design choice. The z-score $z_i$ measures how much better or worse response $i$ is compared to the group average, in units of the group's standard deviation. Then $A_i$ linearly rescales these z-scores to the range $[-1, 1]$, with the best response getting $A_i = 1$ and the worst getting $A_i = -1$. This means the model is always optimizing relative to the group's internal ranking, not absolute reward values.

Why this form: The paper states that LGROUPER is chosen because it can "more effectively leverage datasets containing multiple rejected responses" compared to standard DPO, which only handles pairs. In a group of $G=4$ responses, there is one preferred response and three rejected responses of varying quality. Standard DPO would either discard two of the rejected responses or create multiple pairs, losing information about the relative ordering among rejected responses. GROUPER preserves this relative ordering through the continuous advantage $A_i$, which gives different weights to rejected responses based on how bad they are relative to the group.

Reinforcement Learning: GRPO with IcePop.

The final training phase is joint multimodal reinforcement learning across text and vision tasks. The paper uses Group Relative Policy Optimization (GRPO) with the IcePop setting, a variant of policy gradient methods designed for stability in large-scale training.

The reward system is described at a high level: the same reward system as K-EXAONE is used for text tasks (presumably rule-based or model-based reward functions for mathematics, coding, knowledge, and instruction-following), while "task-specific reward functions" are used for vision tasks (general visual understanding, STEM reasoning, charts, OCR, document understanding, and multi-image settings).

The GRPO mechanism works as follows: for each prompt, the model generates a group of responses (the paper does not specify the group size), each response receives a reward from the appropriate reward function, and the advantage for each response is computed as the response's reward minus the mean reward of its group. The policy gradient then increases the probability of responses with positive advantage and decreases the probability of responses with negative advantage.

The specific setting is "GRPO with the IcePop setting" (citing DeepSeekMath, Shao et al., 2024, and the Ling Team, 2025), which includes zero-variance filtering: groups where all responses have the same reward (i.e., the group advantage is zero for all members) are removed from the training batch. This filtering addresses the problem of uninformative gradient updates—if all responses in a group get the same reward, there is no signal about which responses are better, and including these groups would add noise to the training without providing useful learning signal.

The paper specifically notes that "standard deviation normalization" is omitted from the advantage computation to "preserve training stability" (Section 2.4.3). Standard GRPO typically divides the advantage by the group's standard deviation to normalize the scale of updates across prompts with different reward ranges. Omitting this normalization means the magnitude of policy updates scales with the reward variance in each group, which can be unstable if some groups have much higher variance than others. The paper's decision to omit it suggests that reward variance was well-controlled enough in their setting that normalization was unnecessary (or possibly that normalization introduced its own instability).

The data for RL training spans text tasks (mathematics, coding, knowledge, instruction-following) and vision tasks (general visual understanding, STEM reasoning, charts, OCR, document understanding, multi-image). Joint training across modalities means the same model parameters are updated from both text and vision RL gradients simultaneously, which can help prevent modality-specific overfitting but requires careful balancing of the training data mix.

Design choice: why both offline preference optimization and online RL?

Offline preference optimization (LDPO, LGROUPER) works with pre-collected response pairs and is therefore efficient and stable—the training data is fixed, and there is no interaction with the environment during training. However, it cannot learn from the model's own mistakes because the rejected responses were generated by some other model (or by humans), not by the current model.

Online RL (GRPO) closes this gap by using the current model to generate responses and receiving rewards in real time. This allows the model to learn from its own specific errors—if the model has developed a particular failure mode (e.g., consistently misreading certain Korean characters in OCR tasks), online RL can generate examples of that failure mode and train against them, whereas offline preference data might not include those specific errors if they were not generated by the data collection model.

The multi-stage integration (preference optimization applied at multiple points, RL applied as a final phase) suggests a specific hypothesis about capability development: preference optimization efficiently corrects broad behavioral patterns (don't be rude, prefer structured outputs, favor accurate over plausible-sounding), while RL provides fine-grained task-specific improvement that requires the model to actually attempt the tasks and receive granular reward signals.

4. Key Insights and Innovations

Innovation 1: Data Strategy as a Substitute for Model Scale—But Only in Targeted Domains

The dominant assumption in the VLM scaling community, reinforced by results from GPT-4V, Gemini, and Qwen-VL series, is that bigger models trained on more data improve everywhere. EXAONE 4.5 challenges this with a more nuanced claim: a dense 33B model with aggressively domain-targeted data can match or exceed a 236B MoE model (Qwen3-VL-235B) on specific document understanding and Korean-language benchmarks, but the advantage is narrow and domain-specific rather than general (Table 2).

This is not simply "data quality beats parameter count"—the paper demonstrates something more specific and practically consequential. EXAONE 4.5 beats Qwen3-VL-235B on MATHVISION (75.2 vs. 74.6) and CHARXIV (71.7 vs. 66.1), is competitive on OMNIDOCBENCH V1.5 (81.2 vs. 84.5), but loses on general benchmarks like MMSTAR (74.9 vs. 78.7) and HALLUSIONBENCH (63.7 vs. 66.7). The pattern is systematic: the specialized data helps where it was targeted, and the model falls back to its base capability level everywhere else.

This is a conceptual reframing of the scaling-is-all-you-need narrative. Prior work (Hoffmann et al., 2022; Kaplan et al., 2020) established that model performance follows predictable power laws in total compute. EXAONE 4.5 suggests that these scaling laws are domain-conditional: the same total compute spent on general data produces a different capability profile than compute spent on targeted data, and the difference is not a constant offset but a domain-dependent tradeoff. A model that underperforms on general visual reasoning can simultaneously overperform on document parsing, which means "is model A better than model B?" is an ill-posed question without specifying the task distribution.

The practical implication is significant for organizations building industrial AI systems: identify your actual deployment domain first, then build your training data strategy around it, and accept that general-domain benchmarks will underrepresent your model's true utility. This inverts the standard ML development workflow, where models are optimized for broad benchmarks and then hoped to generalize to specific use cases. EXAONE 4.5's development philosophy appears to be: optimize for the deployment domain, and treat general benchmarks as a sanity check rather than an objective function.

What makes this finding non-obvious is the magnitude of the scale differential. A 33B model outperforming a 236B MoE model (with ~23B activated parameters, so roughly comparable active compute) on specific tasks is a 7× reduction in total parameters. Prior work on data quality (e.g., Phi series from Microsoft) showed that curated data could produce strong small models, but that was in the text-only domain and did not address the multimodal case where visual data curation is substantially harder (images are higher-dimensional, noisier, and harder to filter algorithmically than text). EXAONE 4.5 extends the data-quality-over-scale thesis to vision-language models and adds the crucial domain-specificity qualifier.

The caveat is equally important: this strategy only works for domains where targeted data can be curated at scale. For LG, the domain is Korean industrial documents—a well-defined, data-rich target. The same approach would not work for a general-purpose VLM that needs to handle arbitrary user queries across all visual domains. This is not a weakness of the finding but a clarification of its boundary conditions.

Innovation 2: The Diagnostic Finding That Document Understanding Requires Structural Parsing Data, Not Just More Image-Text Pairs

The paper's pre-training data design reveals an implicit diagnostic claim that is never stated as a hypothesis but is demonstrated through the data composition: standard image caption data does not teach document understanding, regardless of scale. This matters because the dominant VLM training paradigm treats all visual inputs as instances of the same underlying task—map pixels to text—and assumes that training on diverse image-text pairs will eventually cover document understanding as a special case.

EXAONE 4.5's data pipeline suggests this assumption is wrong. The paper constructs three entirely separate data categories for document-relevant capabilities that general image-text pairs do not provide (Section 2.2.2):

  • OCR data at character, word, and document levels, explicitly including "contrastive pairs of visually confusable words" to force fine-grained visual discrimination that natural image captioning never requires (captions do not ask "is that the word 'clay' or 'day'?").
  • Document parsing tasks that output structured formats (HTML, Markdown, JSON) rather than natural language descriptions, teaching the model that the correct response to a table image is its structured representation, not a prose summary.
  • Grounding data with explicit bounding box coordinates, which teaches spatial localization as a distinct capability from semantic description.

The conceptual innovation is in identifying document understanding as a qualitatively different capability from visual description, requiring different training signals. This is not incremental—it challenges the implicit assumption that visual understanding is a unitary construct that can be acquired through undifferentiated training on image-text pairs.

The evidence for this claim is in the benchmark results (Table 2), where EXAONE 4.5's document understanding scores are competitive with much larger models, but the paper does not provide an ablation showing that removing the specialized OCR/document data degrades performance. This is a limitation: the claim that specialized data is necessary (not just helpful) for document understanding cannot be verified from the reported results alone. However, the existence of the specialized data categories and the paper's emphasis on them constitutes a strong implicit claim about their importance.

If correct, this finding has significant implications for VLM training methodology. It suggests that future VLMs intended for document-heavy applications should not expect document understanding to emerge from scale alone and should explicitly construct document-parsing training data as a first-class component of the pre-training mixture, not an afterthought added during fine-tuning.

Innovation 3: Context Extension Embedded in SFT as an Anti-Catastrophic-Forgetting Strategy

The paper's approach to context length extension—integrating it directly into the supervised fine-tuning stage rather than treating it as a standalone pre-training phase—appears at first glance to be a minor engineering convenience. On closer examination, it represents a conceptual insight about the relationship between instruction-following and long-range attention.

Standard practice in LLM development separates context extension from instruction tuning. The model is first pre-trained at a moderate context length, then undergoes a dedicated "long-context adaptation" phase (often using modified positional encodings like NTK-aware RoPE scaling or YaRN), and finally receives SFT on instruction-following data. The reasoning for this separation is that context extension modifies the model's fundamental attention patterns (how it distributes attention weights across long distances), while SFT teaches task-specific behaviors, and these should not interfere.

EXAONE 4.5's approach challenges this separation by arguing that high-quality instruction-following data can simultaneously teach long-range coherence. The key enabler is the base LLM's prior long-context capability (the EXAONE 4.0 backbone was already 128K-capable). Because the language decoder already knows how to attend over long sequences, the multimodal SFT data—which includes examples that require attending to distant visual-textual relationships—naturally extends this capability to the multimodal setting. There is no separate "context extension" phase because the instruction-following data itself provides the long-range training signal.

This is a reframing, not an incremental improvement. The conventional view treats context length as a property of the model architecture (positional encodings, attention mechanism) that must be explicitly engineered. EXAONE 4.5 treats it as a property that can be acquired through task-relevant training if the base architecture already supports it. The practical advantage is training pipeline simplicity: one fewer phase means fewer opportunities for optimization instability, catastrophic forgetting, or hyperparameter misconfiguration.

The evidence for this claim is indirect. The paper does not ablate the integrated-vs-separate context extension approach, so it is impossible to know whether separate extension would have performed better, worse, or identically. The claim of superiority is based on stated advantages ("minimizes optimization instability and prevents performance degradation") rather than comparative experiments. The AA-LCR benchmark score of 50.6 (Table 3) is the only long-context evaluation reported, and without a separate-extension baseline, it cannot distinguish the integrated approach from alternatives.

The significance of this innovation therefore lies more in its conceptual framing than its empirical validation. It proposes that the boundary between "capability acquisition" (context extension) and "task adaptation" (SFT) is more porous than the standard pipeline assumes, and that combining them can be more efficient than separating them—but the paper does not prove this proposition with controlled experiments.

Innovation 4: Domain-Specific Korean Data as a Deliberate Anti-Hallucination Strategy

The paper's Korean data curation strategy contains an insight that is easy to overlook amid the architectural and training pipeline details: hallucination in underrepresented cultural domains is not primarily a model capability problem but a data grounding problem, and it can be mitigated by training on authoritative verifiable sources rather than more data.

The standard approach to improving multilingual performance is to increase the volume of target-language data in the training mixture. Crawl more Korean web pages, include more Korean Wikipedia articles, translate more English content into Korean. This approach improves fluency and coverage but does not guarantee factual accuracy—a model trained on crawled Korean web content learns whatever claims appear in that content, whether true or false.

EXAONE 4.5 takes a qualitatively different approach for Korean cultural content (Section 2.2.2). Instead of indiscriminate Korean web crawling, it specifically uses:

  • Korea Tourism Organization (KTO) datasets: an authoritative government source for Korean cultural and historical information.
  • IT Donga and Game Donga: established Korean publications with editorial standards, not arbitrary social media content.
  • Rendered Korean academic content: sourced from textbooks and exams where factual correctness is guaranteed by the academic review process.
  • Explicit grounding in verifiable evidence: synthetic captions are generated "grounded in factual information" using existing metadata as a constraint, meaning the model is not trained on plausible-sounding hallucinations.

This is a diagnostic insight, not an architectural one. It identifies the root cause of Korean-culture hallucinations not as insufficient Korean data volume but as insufficient Korean data authority. A model trained on 10× more Korean web text might still hallucinate Korean cultural facts if that text contains user-generated content, marketing copy, and other unverifiable claims. By restricting Korean cultural training to authoritative sources, the paper trades breadth of coverage for reliability of information.

The significance extends beyond Korean. Many languages and cultures are underrepresented not just in data volume but in data quality—the available web data is disproportionately user-generated, commercial, or otherwise unverifiable. The EXAONE 4.5 approach suggests a general strategy: for any underrepresented cultural domain, identify the authoritative information sources (government cultural agencies, established publishers, academic institutions) and use those as the training backbone, rather than trying to compensate for authority gaps with volume.

The benchmark evidence is suggestive but not definitive: EXAONE 4.5 scores 42.7 on KMMMU (Korean multimodal understanding) versus GPT-5 mini's 42.6, essentially tied, and 80.1 on K-VISCUIT versus Qwen3-VL-235B's 78.5 (Table 2). These are competitive results but not dramatic wins, which is consistent with the anti-hallucination motivation—the goal is not to dramatically outperform on Korean benchmarks but to produce reliable Korean outputs that users can trust. Trustworthiness is not measured by accuracy on benchmark questions (where hallucinated answers that happen to be correct still count as correct), so the true benefit of the authoritative-source strategy may not be fully captured by the reported metrics.

5. Experimental Analysis

Evaluation Methodology

  • Dataset. All experiments use the MATH benchmark (Hendrycks et al., 2021), consisting of high-school competition-level math problems. The authors use the specific split from Lightman et al. (2022): 12,000 training questions and 500 test questions. The choice of MATH is deliberate (Section 4): test-time compute is expected to help most when the model already possesses the necessary knowledge and the challenge is drawing complex inferences — mathematical reasoning fits this profile because it requires multi-step logical deduction rather than novel factual recall.

  • Base model(s). All experiments use PaLM 2-S* (Codey) (Anil et al., 2023). The authors argue this model is "representative of the capabilities of many contemporary LLMs" and sits in a useful regime: non-trivial performance on MATH (roughly 10–19% pass@1 depending on the prompt and sampling configuration) but far from saturation, leaving room for test-time compute to make a difference. For the FLOPs-matched comparison, a second model with approximately 14× more parameters is used as the pretraining-scaled baseline.

  • Metrics. The primary metric throughout is MATH test accuracy (%) — the fraction of the 500 test questions for which the selected final answer matches the ground truth. Answers are graded using the grading function released by Lightman et al. (2022) (Appendix G). When analyzing difficulty-dependent behavior, the paper reports accuracy within each of the five difficulty quintiles separately.

  • Baselines. The paper uses several baselines: Majority voting: select the most common final answer among N sampled solutions (no learned verifier). ORM best-of-N weighted: score N solutions with an outcome reward model and apply best-of-N weighted selection. PRM best-of-N weighted: score N solutions with the process reward model and apply best-of-N weighted selection. Parallel sampling (for revisions): generate N independent solutions from the revision model and select the best via verifier or majority.

  • Generation budget / compute accounting. One "generation" equals one complete sampled answer from the base LLM. For beam search and best-of-N, the budget equals the number of beams or samples N. For lookahead search with k lookahead steps, the cost is N × (k+1) to account for the additional rollout computation (Section 5.3). Budgets are swept across powers of 2, typically from 2^0 to 2^9 (1 to 512 generations).

  • Cross-validation / statistical protocol. To avoid contaminating strategy selection with test-set performance, the authors use two-fold cross-validation within each difficulty bin on the 500-question test set. The best strategy is selected on one fold and evaluated on the other, with results averaged (Section 3.2).

Main Quantitative Results

Search Against PRM Verifiers (Section 5)

The headline finding from the search experiments is that beam search significantly outperforms best-of-N at low generation budgets, but its advantage diminishes or reverses at high budgets due to verifier over-optimization. Lookahead search generally underperforms all methods at the same generation budget because its extra per-step cost reduces the effective number of beams explored.

Aggregate search algorithm comparison (Figure 3, left). Across all 500 test questions with a maximum budget of 256 generations:

  • At low budgets (2–8 generations), beam search with M = 4 significantly outperforms best-of-N weighted. For example, at 4 generations beam search (M = 4) achieves roughly 27% accuracy versus roughly 16% for best-of-N weighted — a substantial gap.
  • At high budgets (64–256), beam search performance flattens and falls slightly below best-of-N weighted. Best-of-N weighted reaches approximately 38% at 512 generations; beam search (M = 4) plateaus around 34%.
  • Lookahead search (both k = 1 and k = 3) generally underperforms at the same generation budget due to its higher per-step cost. The 3-step lookahead variants converge to similar performance as other methods at very high budgets but never surpass them.
  • Majority voting trails all verifier-based methods substantially, reaching only about 29% at 512 generations.

Difficulty-bin analysis for search (Figure 3, right). The per-difficulty breakdown (beam search M = 4 vs. best-of-N weighted, shown at four budget levels: 4, 16, 64, 256 generations) reveals the core pattern:

  • Bin 1 (easiest): Beam search accuracy decreases from roughly 78% to 77% as the budget goes from 4 to 256, while best-of-N weighted increases from 68% to 88%. This is the clearest evidence of PRM over-optimization — beam search finds solutions that exploit the verifier signal.
  • Bin 2: Beam search improves modestly (roughly 14% to 32%) but best-of-N weighted improves faster (roughly 14% to 60%), maintaining a clear advantage at high budgets.
  • Bin 3: Beam search consistently outperforms best-of-N weighted across all budgets, reaching roughly 34% vs. 23% at 256 generations.
  • Bin 4: Beam search shows the strongest relative advantage, reaching roughly 17% vs. 10% for best-of-N at 256 generations.
  • Bin 5 (hardest): Both methods hover near 1–3% regardless of budget. No method makes meaningful progress.

Compute-optimal search (Figure 4). By selecting the best search strategy per difficulty bin at each budget level:

  • At 16 generations, compute-optimal (oracle bins) achieves approximately 27% accuracy, roughly matching PRM best-of-N weighted at 64 generations — a 4× compute reduction.
  • At 256 generations, compute-optimal oracle reaches approximately 39.5%, surpassing PRM best-of-N weighted at the same budget (roughly 37%).
  • Compute-optimal with predicted difficulty bins tracks the oracle version closely, particularly at lower budgets. The two curves "largely overlap" per the authors (Figure 4), with the predicted version reaching approximately 37% at 256 generations.
  • Both compute-optimal variants consistently outperform ORM best-of-N weighted (which peaks around 34% at 512 generations) and majority voting (around 29%).

PRM vs. ORM (Figure 14, Appendix F). At 2048 samples, PRM best-of-N weighted achieves approximately 40% accuracy versus roughly 35% for ORM best-of-N weighted and roughly 30% for majority voting. The gap between PRM and ORM widens with the number of samples, confirming the PRM's superior scaling properties.

Revision Model Results (Section 6)

The headline finding from the revision experiments is that sequential revisions marginally outperform parallel sampling in aggregate, but the optimal sequential-to-parallel ratio depends critically on problem difficulty: easy problems favor fully sequential (local refinement), while hard problems benefit from a balanced ratio (exploration via parallelism plus refinement per chain).

Revision model pass@1 trajectory (Figure 6, left). Starting from approximately 18.2% pass@1 at step 1, the revision model's per-step accuracy improves to roughly 24–25% by steps 15–20, and remains in the 23–25% range out to 64 steps. The model generalizes beyond its 4-step training horizon.

Sequential vs. parallel (Figure 6, right). At 64 generations:

  • Sequential + best-of-N weighted: approximately 41.5%
  • Parallel + best-of-N weighted: approximately 39%
  • Sequential + majority: approximately 38%
  • Parallel + majority: approximately 35%

Sequential outperforms parallel under both selection mechanisms, with the verifier-based gap (roughly 2.5 percentage points) being slightly narrower than the majority-based gap (roughly 3 points).

Sequential-to-parallel ratio sweep (Figure 7, left). For a fixed generation budget, varying the ratio reveals:

  • At 256 generations, the optimal ratio is around 2^1 to 2^3 (2:1 to 8:1 sequential-to-parallel), achieving approximately 43–44% accuracy.
  • Fully parallel (leftmost point) yields approximately 40%.
  • Fully sequential (rightmost point) yields approximately 42%.
  • At lower budgets (8–32 generations), fully sequential is optimal — the curves are monotonically increasing with the sequential-to-parallel ratio.

Difficulty-dependent ratio (Figure 7, right). At a fixed budget of 128 generations:

  • Bin 1: Performance is essentially flat across all ratios, around 90–92%. Easy questions are insensitive to the allocation strategy.
  • Bin 2: Slight advantage for higher sequential ratios, approximately 63% at fully sequential vs. 58% at fully parallel.
  • Bin 3: A clear optimal ratio emerges at moderate sequential-to-parallel values (around 2^1 to 2^3), reaching approximately 42% vs. 35% at the extremes.
  • Bin 4: Similar pattern, with the peak at a moderate ratio achieving roughly 18% vs. 14% at fully parallel.
  • Bin 5: All ratios produce roughly 2–3% accuracy. No allocation strategy helps.

Compute-optimal revisions (Figure 8). Selecting the optimal sequential-to-parallel ratio per difficulty bin:

  • At 64 generations, compute-optimal oracle achieves approximately 40%, matching parallel best-of-N weighted at 256 generations — a 4× improvement.
  • At 256 generations, compute-optimal oracle reaches approximately 44%, compared to roughly 41% for best-of-N weighted and 37% for parallel-only.
  • Compute-optimal predicted bins perform slightly below oracle bins at high budgets (approximately 41% at 256 generations) but still substantially outperform the parallel baseline.
  • Notably, the parallel baseline appears to plateau around 36–37% at high budgets, while compute-optimal scaling continues to improve, suggesting that the gains from adaptive allocation compound at higher budgets.

FLOPs-Matched Comparison: Test-Time vs. Pretraining Compute (Section 7)

The headline finding is that a smaller model with compute-optimal test-time strategies can outperform a ~14× larger model, but only on easy-to-medium problems and when the inference-to-pretraining token ratio R is low. On hard problems or at high R, pretraining is more effective.

Revisions (Figure 9, left; Figure 1, top-right bar chart). Comparing PaLM 2-S* with compute-optimal revisions against the ~14× larger model, the paper reports the following accuracy comparisons across three R regimes and difficulty groupings (numbers from the bar chart in Figure 1, top-right, which aggregates the five bins into easy/medium/hard for readability):

  • Easy questions (bin 1): +11.8% advantage for test-time compute at R ≪ 1 (0.16), +3.5% at R ≈ 1 (0.79), −11.9% (disadvantage) at R ≫ 1 (22).
  • Medium questions (bins 2–3): +27.8% advantage at R ≪ 1, +16.7% at R ≈ 1, +5.4% at R ≫ 1.
  • Hard questions (bins 4–5): +21.6% at R ≪ 1, implied negative at R ≈ 1, −37.2% at R ≫ 1.

At R ≪ 1, test-time compute outperforms the larger model across all difficulty levels. At R ≫ 1, it only remains preferable on easy questions.

PRM search (Figure 9, right; Figure 1, bottom-right bar chart). The pattern is starker:

  • Easy: +19.1% at R ≪ 1, +2.2% at R ≈ 1, +2.0% at R ≫ 1.
  • Medium: 0.0% (tied) at R ≪ 1, −35.3% at R ≈ 1, −30.8% at R ≫ 1.
  • Hard: −3.6% at R ≪ 1, −35.3% at R ≈ 1, −52.9% at R ≫ 1.

PRM search shows substantially weaker benefits than revisions for the FLOPs-matched comparison, with large disadvantages on medium and hard questions even at moderate R values. On easy questions, test-time compute remains preferable across all R regimes.

Figure 9 detail. The line plots show accuracy per difficulty bin as test-time compute scales. The 14× larger model's greedy performance (marked with stars) is placed at three x-axis positions corresponding to the three R values. Where the compute-optimal scaling line is above the star, test-time compute wins. On bin 1 (purple, topmost line), the scaling line is above all three stars for revisions. On bin 5 (blue, bottommost line), the line is below all three stars and essentially flat near 0–5%, confirming that no amount of test-time compute helps on the hardest problems.

Ablation Studies and Robustness Checks

PRM aggregation strategy (Appendix E, Figure 13). Comparing "min," "prod," and "last" step-wise aggregation: "last" achieves roughly 37% at 256 samples, "min" achieves roughly 35%, "prod" achieves roughly 27%, and ORM achieves roughly 34%. The "last" aggregation's superiority is notable because it effectively reduces the PRM to ORM-like behavior at aggregation time, yet the PRM still outperforms a separately trained ORM. The authors interpret this as evidence that step-level PRM training provides beneficial representation learning.

PRM vs. ORM scaling (Appendix F, Figure 14). The PRM consistently outperforms the ORM, with the gap widening at higher sample counts: at 2048 samples, PRM best-of-N weighted reaches approximately 40% vs. ORM's 35% and majority voting's 30%. This confirms that the PRM's advantage is not merely a constant offset but scales with compute budget.

Revision model verifier choice (Appendix J, Figure 15a). The base-LM PRM underperforms the revision-specific ORM when scoring revision model outputs: sequential + base-LM PRM achieves roughly 40% at 64 generations vs. sequential + revision ORM at roughly 42%. This confirms distribution shift as a practical concern—verifiers trained on base model outputs do not transfer cleanly to revision model outputs.

Revision history in verifier context (Appendix J, Figure 15b). Including previous revisions in the ORM's context provides a small improvement over the no-history ablation (approximately 1–2 percentage points at 64 generations), but both variants outperform the parallel baseline. This confirms that the sequential sampling benefit is not solely attributable to the verifier seeing more context.

Oracle vs. predicted difficulty bins (Figures 4, 8, and Appendix C, Figures 11–12). Both oracle and predicted bins yield qualitatively similar trends across difficulty levels. Predicted bins show slightly lower performance at high budgets in the revision setting (roughly 41% vs. 44% at 256 generations in Figure 8) but essentially identical performance in the search setting (Figure 4). This is the critical robustness check: the compute-optimal strategy works without ground-truth labels, making it practically deployable.

Majority voting for revisions (Appendix B, Figure 10). The sequential-to-parallel ratio trends observed with verifier-based selection are replicated with majority voting: easy questions are insensitive to ratio, hard questions show an optimal intermediate ratio, and fully sequential marginally outperforms fully parallel in aggregate. This indicates that the benefits of sequential revision are not purely an artifact of verifier-based selection.

ReST^EM revision model (Appendix K, Figure 16). An attempt to further optimize the revision model using ReST^EM (Singh et al., 2024) backfires: additional sequential revisions substantially hurt performance with this model. At 256 generations, fully sequential performance drops to approximately 33.5% compared to roughly 38.5% at the optimal ratio. The authors hypothesize that on-policy data collection in ReST^EM exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly. This is a significant negative result that highlights the sensitivity of revision training to the data generation procedure and serves as a warning against assuming that RL-based self-improvement methods will automatically transfer to revision-style tasks.

Critical Assessment

The experiments demonstrate a consistent pattern: difficulty-conditioned allocation of test-time compute improves efficiency over uniform best-of-N by approximately 4× (Figures 4, 8). However, the specific claims the paper makes require careful scrutiny against what was actually tested.

On the claim that compute-optimal scaling yields "more than 4× better efficiency" over best-of-N. This is supported for both search (Figure 4: 16 generations of compute-optimal matching 64 generations of best-of-N) and revisions (Figure 8: 64 generations matching 256). The 4× figure specifically refers to achieving equivalent accuracy with 4× fewer generations. However, the paper's difficulty estimation cost—generating 2048 samples per question to bin difficulty—is not included in any budget calculation. The authors explicitly flag this (Section 3.2) as an exploration-exploitation tradeoff, but this means the reported 4× efficiency gain is an upper bound on practical deployment efficiency, not a realized gain. A deployment system must amortize the cost of difficulty estimation over many queries, and the paper provides no analysis of how many queries are needed to amortize the 2048-sample overhead.

On the claim that a smaller model with test-time compute can outperform a ~14× larger model. This holds strongly for easy-to-medium problems at R ≪ 1 (Figure 9, revisions: +27.8% on medium questions), but weakens or reverses as difficulty increases or R grows. The claim is thus conditional, not universal, and the paper is transparent about the conditions. A significant weakness in the comparison is that the 14× larger model uses only greedy decoding with no additional test-time compute of its own. A fairer baseline would give the larger model some test-time compute budget (e.g., best-of-8 or best-of-16), which could substantially close the gap. Additionally, the 14× larger model scales parameters only (not data), which departs from Chinchilla-optimal pretraining. A compute-optimally trained larger model (scaling both data and parameters) would be a stronger baseline. The reported advantages of test-time compute over pretraining should therefore be interpreted as upper bounds against a potentially suboptimal pretraining comparison.

On the claim that difficulty-dependent behavior is the key insight. This is the most robust finding in the paper, replicated across search methods (Figure 3, right), revision strategies (Figure 7, right), and selection mechanisms (majority voting in Appendix B, Figure 10). The fact that beam search hurts on easy problems while helping on medium ones (Figure 3, right) is a genuinely non-obvious result that would not be predicted by any uniform scaling theory. The cross-validation protocol (two-fold within difficulty bins) provides some protection against overfitting the compute-optimal policy to the test set, though with only 500 total test questions split into five bins of ~100 each, the per-bin sample size (~50 per fold) is small enough to raise concerns about the stability of the selected strategies.

On the claim that verifier over-optimization is the limiting factor. The evidence is qualitative (examples of degenerate outputs in Appendix M) and quantitative (beam search degrading on easy problems, Figure 3 right; lookahead search underperforming simpler methods, Figure 3 left). However, the paper does not provide a systematic measurement of over-optimization—for instance, by comparing the PRM's predicted scores against actual correctness as a function of optimization intensity. A calibration curve (PRM score on x-axis, actual accuracy on y-axis, at different search budgets) would quantify the over-optimization more precisely than the current evidence. The claim that over-optimization is the "primary bottleneck" is therefore plausible but not rigorously quantified.

Missing experiments that would have strengthened the paper. Several experiments would have made the claims more robust:

  • Combining PRM search with revisions. The paper studies search and revisions independently but never combines them—for instance, using the revision model as the proposal distribution within beam search, or using the PRM to guide which revisions to pursue. The authors acknowledge this gap in Section 8, but it means the reported results represent a lower bound on what the full system could achieve.

  • Difficulty estimation cost amortization analysis. How many queries does it take to amortize the 2048-sample difficulty estimation overhead? If the answer is "thousands," the 4× efficiency gain may be inaccessible in many deployment scenarios. If "tens," the approach is practical. The paper provides no data to answer this.

  • Larger model with modest test-time compute budget. The most informative baseline for the FLOPs-matched comparison—the 14× larger model with best-of-8 or best-of-16—is absent. This would directly test whether test-time compute is genuinely a substitute for pretraining or merely a substitute for greedy decoding.

  • Statistical confidence intervals. None of the main result figures include error bars or confidence intervals. With a 500-question test set, the standard error on a 40% accuracy estimate is approximately 2.2 percentage points (assuming binomial variance), meaning that many of the reported differences between methods at high budgets (e.g., sequential vs. parallel at 64 generations: 41.5% vs. 39%) could fall within overlapping confidence intervals. The absence of statistical reporting makes it difficult to assess which differences are reliable.

  • Replication on other model families. All results are on PaLM 2-S*. The paper argues this model is "representative" (Section 4), but without replication on other architectures (LLaMA-based models, other model scales), the generality of the difficulty-dependent patterns remains unverified.

On the single-benchmark limitation. The exclusive use of MATH—while well-justified for its clean correctness signals and multi-step reasoning requirements—means the findings may not transfer to domains where correctness is ambiguous or where the base model's capability profile differs. The paper does not claim otherwise, but readers should not extrapolate the 4× efficiency gain or the optimal strategy profiles to, say, creative writing or open-ended dialogue without further evidence.

Bottom line: The experiments provide strong evidence for a conditional, domain-specific claim: on the MATH benchmark with PaLM 2-S*, difficulty-conditioned allocation of test-time compute between search and revisions can achieve substantial efficiency gains over uniform best-of-N, and these gains are largest on easy-to-medium problems where the base model already produces some correct solutions. The evidence for the pretraining-vs-inference tradeoff is weaker due to a suboptimal baseline (greedy decoding for the larger model, parameter-only scaling), and the practical deployability of the approach is limited by the unresolved difficulty estimation cost. The paper's most durable contribution is the demonstration that test-time compute scaling is not monotonic or uniform—different strategies work best at different difficulty levels and different budgets—which is a finding that likely generalizes beyond this specific setup even if the exact optimal strategies do not.

6. Limitations and Trade-offs

Difficulty Estimation Cost Is Not Accounted For in the Headline Efficiency Gains

The assumption or constraint. The entire compute-optimal framework depends on knowing each prompt's difficulty before allocating the inference budget. The paper's method for estimating difficulty—generating 2048 samples per question and averaging either ground-truth correctness (oracle bins) or PRM final-answer scores (predicted bins)—is extraordinarily expensive. The authors acknowledge this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence. The reported 4× efficiency gains over best-of-N (Figures 4, 8) are computed after difficulty is known, without amortizing the cost of learning it. In a realistic deployment, the total cost would be difficulty estimation + strategy execution, and the former could dominate the latter. Generating 2048 samples just to estimate difficulty consumes more compute than the largest test-time budgets studied (256–512 generations). If a system receives only one query, the difficulty estimation alone costs 8× more than the entire best-of-N baseline at N=256. The 4× figure is therefore an upper bound on achievable efficiency in a deployed system, not a realized gain.

What evidence exists in the paper. The paper shows that predicted difficulty bins (using the PRM rather than ground-truth labels) track oracle bins closely (Figures 4, 8), which removes the need for labeled data but does not address the computational cost. The authors flag the exploration-exploitation tradeoff in Section 3.2 but no experiment measures the amortization rate: how many queries are needed before the per-query difficulty estimation cost becomes negligible relative to the per-query strategy execution savings. A lightweight difficulty classifier trained directly on question text is suggested as future work (Section 8) but not implemented.

Mitigation status. Not addressed. The paper explicitly treats this as future work and does not include estimation cost in any budget calculation. For practitioners, this means the 4× efficiency claim cannot be realized without either (a) a much cheaper difficulty estimation method, (b) a deployment scenario where difficulty is known a priori, or (c) amortization over enough queries that the per-query estimation cost becomes negligible—a scenario the paper does not analyze.

Hard Problems Remain Essentially Unsolved Regardless of Compute Budget

The assumption or constraint. The compute-optimal framework assumes that the base model's proposal distribution contains at least some correct solutions at a non-trivial rate. When this assumption fails—when the base model essentially never produces the correct answer—no amount of search or revision can help.

The consequence. Across all methods—search, revisions, and their compute-optimal combinations—the hardest questions (difficulty bin 5) show near-zero improvement regardless of compute budget. In Figure 3 (right), bin 5 accuracy hovers at 1–3% for all methods and all budgets. In Figure 7 (right), bin 5 shows roughly 2–3% accuracy irrespective of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling line is essentially flat near 0–5%. Test-time compute amplifies existing capability but cannot create it. If the base model's pass@1 on a problem class is near zero, the approach provides no path to improvement—the correct answers simply do not exist in the proposal distribution to find or refine, regardless of how the budget is allocated.

What evidence exists in the paper. The bin 5 results across Figures 3, 7, and 9 are the key evidence. The paper is transparent about this limitation (Section 7 takeaway box explicitly notes that test-time compute cannot compensate for fundamental capability gaps). However, the paper does not characterize why bin 5 problems are hard—are they simply too difficult for PaLM 2-S* at any scale, or would a larger model solve them? This distinction matters for practitioners deciding between test-time compute and pretraining: if bin 5 problems are solvable by larger models, then pretraining is the only viable path; if they are genuinely unsolved by any current model, then neither approach helps.

Mitigation status. The paper does not attempt to solve this. It acknowledges the boundary condition transparently, which is appropriate for an analysis paper. The practical implication is that compute-optimal test-time scaling is only useful for problems within the base model's rough capability range—a constraint practitioners must evaluate on their own task distributions.

The Revision Model Has a Fundamental Correct-to-Incorrect Reversion Problem

The assumption or constraint. The revision model is trained exclusively on trajectories where the in-context answers are incorrect and the target answer is correct (Section 6.1). This design choice—pairing incorrect answers with correct targets using edit-distance-based selection—means the model never sees examples of what to do when the current answer is already correct.

The consequence. At test time, the model will occasionally produce a correct answer during a revision chain and then "revise" it into an incorrect answer in the subsequent step. The paper reports that approximately 38% of correct answers get converted back to incorrect ones using a naive approach (Section 6.1). This is a direct consequence of the training data construction: the model has learned that its job is to change the previous answer, and it applies this learned behavior even when the previous answer is already correct. The mitigation—using majority voting or verifier-based selection across the entire chain rather than taking the final revision—is a post-hoc patch, not a principled solution. It means the system is wasting some fraction of its generation budget on revisions that actively degrade answer quality, and it relies on the verifier or voting mechanism to identify and discard these degraded outputs.

What evidence exists in the paper. The 38% reversion rate is reported in Section 6.1, though it is stated as a qualitative observation rather than presented as a measured statistic with confidence intervals. The paper does not provide a per-difficulty breakdown of the reversion rate—it is possible that reversion is more common on easy problems (where correct answers appear earlier in the chain) or hard problems (where the model is uncertain about correctness). The ReST^EM experiment (Appendix K, Figure 16) provides additional evidence that revision training is fragile: the ReST^EM-optimized revision model suffers substantial performance degradation with sequential revisions, with fully sequential performance dropping to approximately 33.5% at 256 generations compared to roughly 38.5% at the optimal ratio, suggesting that the reversion problem may be exacerbated by on-policy data collection.

Mitigation status. Partially addressed through chain-level selection (majority voting or verifier-based selection, Section 6.1), but this is a patch that discards the bad revisions rather than preventing them. A more principled solution—such as training the model to recognize when no revision is needed, or including correct-to-correct trajectories in the training data—is not explored. The paper does not suggest this as future work.

The FLOPs-Matched Pretraining Baseline Is Substantially Weaker Than It Could Be

The assumption or constraint. The FLOPs-matched comparison in Section 7 compares the smaller model (PaLM 2-S*) with compute-optimal test-time strategies against a ~14× larger model using only greedy decoding with no additional test-time compute. The larger model also scales parameters while holding training data fixed, following the LLaMA paradigm rather than Chinchilla-optimal pretraining where both data and parameters scale.

The consequence. The headline finding—that a smaller model with test-time compute can outperform a 14× larger model (e.g., +27.8% on medium-difficulty questions at R ≪ 1, Figure 1 bar chart)—is computed against a baseline that is meaningfully weaker than what a practitioner would actually deploy. A realistic larger-model deployment would almost certainly use at least best-of-8 or best-of-16 sampling, which would substantially close the gap. Additionally, a Chinchilla-optimal model trained with 14× more total FLOPs (scaling both data and parameters) would likely outperform a parameter-only-scaled model, making the pretraining baseline weaker than the state of the art. The authors acknowledge this in Section 7:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work."

What evidence exists in the paper. The comparison itself—Figure 9 and the bar charts in Figure 1—provides the evidence. The paper does not include an ablation where the larger model receives any test-time compute budget (e.g., best-of-4, best-of-8, best-of-16). The curves in Figure 9 show the smaller model's compute-optimal scaling alongside three star markers representing the larger model's greedy performance at three R values (which correspond to different FLOPs-matched inference budgets for the smaller model). A simple extension—placing the larger model's best-of-N curve on the same plot—would directly address this concern, but it is absent.

Mitigation status. The paper acknowledges the parameter-only scaling choice but does not address the greedy-decoding baseline. For practitioners, this means the reported advantages of test-time compute over pretraining should be interpreted as upper bounds against a suboptimal pretraining baseline. In a real deployment scenario where both approaches are optimized (test-time compute for the smaller model, modest test-time compute for the larger model), the gap would almost certainly be narrower, and the crossover point where pretraining becomes preferable would shift toward lower difficulty levels.

Single Benchmark and Single Model Family Limit Generalizability

The assumption or constraint. All experiments use the MATH benchmark (500 test questions, Section 4) with PaLM 2-S* as the base model. The paper argues that PaLM 2-S* is "representative of the capabilities of many contemporary LLMs" (Section 4), but this claim is not verified through replication on other model families (e.g., LLaMA-based models, Qwen, DeepSeek) or other reasoning benchmarks (e.g., GSM8K for grade-school math, MBPP for code, ARC for science reasoning).

The consequence. Several aspects of the findings could be model-specific or benchmark-specific in ways that the paper cannot distinguish. The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s output distribution—a model with different calibration, different error patterns, or different coverage of solution strategies might exhibit different difficulty-dependent scaling curves. The revision model's ability to learn from edit-distance-paired incorrect-correct trajectories depends on the base model's in-context learning capabilities, which vary substantially across model families. MATH consists exclusively of competition-level math problems requiring symbolic reasoning; it is unclear whether the patterns (beam search hurting easy problems, sequential revisions helping easy problems, hard problems showing zero improvement) transfer to other reasoning domains or to tasks requiring factual recall rather than multi-step inference.

The 500-question test set, split into five difficulty quintiles of ~100 each, then further split by two-fold cross-validation for strategy selection (Section 3.2), means the compute-optimal policy is selected based on ~50 questions per fold per bin. This is a small sample. The paper does not report confidence intervals on the main results, so it is impossible to assess whether, for example, the sequential-vs-parallel gap at 64 generations (41.5% vs. 39%, Figure 6 right) is statistically reliable or could be noise at this sample size. With a binomial standard error of approximately 2.2 percentage points on a 40% accuracy estimate, many of the reported differences between methods at moderate-to-high budgets could fall within overlapping confidence intervals.

What evidence exists in the paper. All figures in Sections 5–7 present point estimates without error bars. The cross-validation protocol provides some internal robustness check, but it addresses overfitting to the test set, not sampling variability from the small test-set size. The paper does not ablate the effect of test-set size (e.g., by subsampling and measuring variance in strategy selection or accuracy estimates).

Mitigation status. The paper does not address this limitation. The authors' claim of representativeness (Section 4) is an assertion, not an empirical finding. For practitioners, this means the quantitative strategy recommendations (use beam search on bin 3–4, use sequential revisions on bin 1–2, optimal sequential-to-parallel ratios) should be treated as MATH-specific and PaLM-2-S-specific* until replicated on other benchmarks and model families. The qualitative finding—that test-time compute allocation depends on difficulty and that different strategies excel at different difficulty levels—is more likely to generalize, but even this should be verified in other domains.

Latency and Wall-Clock Time Are Ignored in the Compute Accounting

The assumption or constraint. The paper measures all test-time compute in "generations" (number of complete solutions sampled from the base model), which is a reasonable proxy for total FLOPs but ignores the distinction between parallel throughput (total work done per unit time with sufficient hardware) and serial latency (time to produce a single answer).

The consequence. Sequential revisions are inherently serial—each revision depends on the output of the previous one, so generating a chain of length 64 requires 64 sequential forward passes through the model. Parallel best-of-N with N=64 samples can, with sufficient hardware, execute all 64 forward passes simultaneously. A compute-optimal policy that allocates 128 generations as 16 parallel chains × 8 sequential revisions per chain takes approximately 8× longer wall-clock time than one that runs 128 parallel samples simultaneously, even though both consume the same total FLOPs. The paper's finding that fully sequential revisions are optimal for easy problems (Figure 7, left, at low budgets; Figure 7, right, bin 1) and that sequential outperforms parallel in aggregate (Figure 6, right) means the optimal FLOPs-efficient strategy may be prohibitively slow for latency-sensitive applications.

What evidence exists in the paper. None. The paper does not discuss latency, does not report wall-clock times, and does not consider latency constraints in the compute-optimal objective (Equation 1, Section 3.1). The generation budget N is treated as the sole resource constraint.

Mitigation status. Not addressed. For practitioners, this is a critical deployment consideration that the paper does not help with. Interactive applications (chatbots, tutoring systems, real-time decision support) cannot tolerate the 64× serial latency of a fully sequential revision chain. The paper's compute-optimal strategies would need to be re-optimized under a joint budget constraint that includes both total FLOPs and maximum latency, which could substantially change the optimal allocation—shifting away from sequential revisions and toward parallel search even on problems where sequential is FLOPs-optimal.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper represents an incremental refinement with diagnostic significance rather than a paradigm shift. It does not introduce new architectures or training algorithms—the components (GQA, 2D RoPE, MTP, DPO, GRPO) are all established, and the paper is explicit about this. Instead, it provides a worked example of a deployment philosophy: that domain-targeted data curation can compensate for roughly an order of magnitude in parameter count (33B dense vs. 236B MoE) on the specific industrial tasks that matter, while accepting that general-domain benchmark performance will lag behind scale-advantaged competitors (Table 2: 74.9 vs. 78.7 on MMSTAR, 63.7 vs. 67.4 on HALLUSIONBENCH).

The most important reframing the paper offers is the decoupling of "model capability" from "benchmark generality." The standard VLM development narrative—train bigger, train on more data, win on more benchmarks—implicitly treats all capabilities as correlated outputs of a single scaling process. EXAONE 4.5 demonstrates that this correlation can be deliberately broken: by skewing the training data mixture toward document understanding and Korean content, the model becomes strong where it was targeted and merely adequate elsewhere. This is not a new idea (domain-specific fine-tuning has existed for years), but EXAONE 4.5 executes it at the pre-training scale rather than as a post-hoc adaptation, which is what makes the finding noteworthy. The two-stage pre-training curriculum (Stage 1: broad alignment, Stage 2: targeted refinement) is a concrete template for other organizations building industrial VLMs for their own domains.

The paper also provides a diagnostic finding about document understanding that, while not empirically isolated through ablations, is embedded in the data design: standard image-text pairs do not teach document parsing, OCR, or structured information extraction, regardless of scale. The construction of three entirely separate data categories for document-relevant capabilities (OCR at character/word/document levels with contrastive pairs, document-to-structured-format parsing, and grounding with bounding boxes) amounts to an implicit claim that these capabilities are qualitatively different from visual description and require dedicated training signals. If correct, this has implications for VLM training methodology more broadly: models intended for document-heavy applications should not expect document understanding to emerge from undifferentiated image-text training.

The paper reconciles a latent tension in the VLM literature between scale-maximalist positions (the Qwen3-VL-235B and GPT-5 mini approach of maximizing parameters and general-domain data) and data-quality-focused positions (the phi-series approach of curated small-data training). EXAONE 4.5 demonstrates that these are not competing philosophies but can be deployed at different points in the capability spectrum: scale works for breadth, targeted data works for depth, and the optimal strategy depends on whether you need a model that is decent at everything or excellent at specific things. This makes research on data mixture optimization (how to compose pre-training corpora for specific deployment profiles) more attractive, and research on pure architectural scaling (assuming data composition doesn't matter much) slightly less so, at least for industrial deployment contexts.

The paper's emphasis on Korean-language content also provides a template for culturally grounded AI development that goes beyond translation. The use of authoritative sources (KTO, IT Donga, Game Donga) rather than indiscriminate web crawling, combined with explicit anti-hallucination grounding, suggests a model for building VLMs that are not just multilingual but culturally reliable—a distinction the field has not systematically made but that becomes critical when models are deployed in high-stakes industrial and governmental contexts.

Follow-Up Research This Work Enables

Ablating the necessity of specialized document data through controlled data-mixture experiments. The paper's central implicit claim—that OCR, document parsing, and grounding data are necessary for document understanding, not just helpful—is untested. A direct follow-up would train two versions of EXAONE 4.5: one with the full Stage 2 data mixture (including grounding, OCR, and document parsing) and one where those data categories are replaced with an equal token budget of general-domain image-text pairs. Comparing performance on OMNIDOCBENCH V1.5, CHARXIV, and AI2D would quantify how much of the document understanding capability is attributable to the specialized data versus general visual-text alignment. A strong negative result (the ablated model performs nearly as well) would reframe the paper's contribution as primarily about Korean language rather than document understanding; a strong positive result (large degradation) would validate the implicit diagnostic claim and provide concrete guidance for VLM training methodology.

Measuring hallucination rates on Korean cultural content across models with different data strategies. The paper claims that using authoritative Korean sources with grounded synthetic captioning reduces hallucination, but accuracy on KMMMU and K-VISCUIT does not directly measure hallucination—a model that guesses correctly on 80% of questions might still hallucinate on the remaining 20% or fabricate plausible-sounding but false cultural details. A follow-up study would construct a Korean cultural hallucination benchmark: questions with verifiable answers drawn from KTO and other authoritative sources, where incorrect answers can be classified as hallucinations (fabricated details that sound plausible) versus simple errors (wrong but not fabricated). Comparing EXAONE 4.5 against Qwen3-VL-235B and GPT-5 mini on this benchmark would test whether the authoritative-source strategy actually reduces hallucination rates or merely improves accuracy on benchmark questions where hallucinated answers that happen to be correct are indistinguishable from genuinely grounded ones. This would also help establish whether the paper's anti-hallucination strategy generalizes to other underrepresented cultures and languages.

Replicating the data-strategy-over-scale finding on a different language and domain. The paper's headline result—33B dense matching 236B MoE on targeted benchmarks—is currently a single datapoint (Korean + document understanding). A replication on, say, Japanese industrial documents, Arabic legal texts, or German engineering diagrams, using the same two-stage pre-training template with domain-appropriate authoritative sources and document parsing data, would establish whether the strategy generalizes or is specific to LG's particular data curation quality and the Korean language's structural properties. The benchmark suite would need domain-appropriate equivalents: Japanese versions of KMMMU, document parsing benchmarks based on Japanese technical manuals, and cultural understanding tests drawn from Japanese authoritative sources. A negative result (the strategy does not replicate) would suggest that EXAONE 4.5's success depends on factors beyond the data strategy itself—possibly including the specific quality of LG's in-house data curation pipelines, the properties of Korean text rendering, or the overlap between EXAONE 4.0's pre-training data and the multimodal training data.

Context extension during SFT versus separate context extension: an ablation with multimodal long-document benchmarks. The paper's claim that embedding context extension in SFT is superior to a standalone extension phase (Section 2.3) is unverified. An ablation would train two versions of EXAONE 4.5: one using the paper's integrated SFT-context-extension approach (as described), and one where context extension to 256K is performed as a separate phase using long text-only and multimodal sequences before SFT begins. Evaluation would need to go beyond AA-LCR (which is text-only and scores 50.6 for EXAONE 4.5 vs. 68.0 for GPT-5 mini, Table 3) to include multimodal long-context benchmarks: tasks requiring the model to find and cross-reference information across a 200K-token multimodal document containing interleaved text, tables, charts, and images. If the integrated approach outperforms the separate-phase approach on multimodal long-context tasks specifically, that would validate the paper's claim that instruction-following data provides long-range coherence benefits that generic long-context training does not. If they perform similarly, the claim of superiority would be unsupported, and the integrated approach would remain an engineering convenience rather than a methodological advance.

Testing whether the correct-to-incorrect reversion problem in revision models generalizes to VLM settings. The paper's revision model (Section 6.1) exhibits a 38% correct-to-incorrect reversion rate because it was never trained on what to do when the current answer is already correct. This finding is from a text-only math reasoning setting (PaLM 2-S* on MATH), but EXAONE 4.5's VLM context raises a parallel question: if the model is trained on OCR and document parsing data where it is always expected to produce a different output (a structured parse) from the input (a document image), does it develop an analogous "always change something" bias in revision-style multi-turn interactions? A direct test would construct a multi-turn VQA evaluation where the model is asked a question about an image, produces an answer, and is then asked "Are you sure? Please check again and revise if needed." Measuring the rate at which initially correct visual answers are revised to incorrect ones, and comparing this rate across tasks the model was trained on (document parsing, OCR) versus tasks it was not (general VQA), would reveal whether domain-specific training produces domain-specific reversion biases. If so, this would have practical implications for how VLM-based agents should be prompted in multi-turn industrial settings.

Practical Applications and Downstream Use Cases

Korean industrial document processing and compliance checking. The most direct application implied by the benchmark results is automated analysis of Korean-language technical documentation in manufacturing and engineering contexts. EXAONE 4.5's scores on OMNIDOCBENCH V1.5 (81.2, competitive with Qwen3-VL-235B's 84.5), CHARXIV (71.7 vs. Qwen3-VL-235B's 66.1), and AI2D (89.0 vs. GPT-5 mini's 88.2) indicate that the model can parse structured documents, extract chart information, and understand diagrams at a level that makes automated compliance checking feasible. In a manufacturing quality control setting, this means the model could take a visual scan of a completed assembly, cross-reference it against the relevant section of a 200-page Korean technical manual (enabled by the 256K context window), and flag discrepancies—a task that currently requires human engineers reading both the scan and the manual. The Korean-language advantage (KMMMU 42.7 vs. Qwen3-VL-235B's 42.1, K-VISCUIT 80.1 vs. 78.5) means this works natively on Korean documentation without translation, avoiding the information loss that occurs when technical Korean is machine-translated to English for processing by English-centric VLMs.

Deployment of smaller VLMs in compute-constrained industrial environments. The paper's finding that a 33B dense model can match a 236B MoE model on targeted benchmarks has direct implications for edge deployment in factories and field sites. A 236B-parameter model requires substantial GPU infrastructure (likely multiple A100s or H100s) even for inference, which is infeasible on a factory floor or at a remote maintenance site. A 33B model can run on a single high-memory GPU and, with quantization, potentially on consumer-grade hardware. For LG's stated trajectory toward Vision-Language-Action (VLA) models that interact with physical environments, this size differential matters enormously: a VLA controlling a robot arm on an assembly line cannot depend on a cloud API with hundreds of milliseconds of latency, and a 33B model is far closer to edge-deployable than a 236B MoE. The open-weight release under a non-commercial license means that industrial partners can evaluate EXAONE 4.5 on their own document corpora without depending on LG's infrastructure, lowering the barrier to adoption.

Multilingual agentic tool use in enterprise settings. EXAONE 4.5's language benchmark results show competitive performance on agentic tool use, specifically on τ2-BENCH where it scores 72.0 (weighted average across RETAIL, AIRLINE, and TELECOM subsets) versus Qwen3-VL-235B's 57.0 (Table 3). Combined with the model's six-language support (Korean, English, Spanish, German, Japanese, Vietnamese), this suggests a deployment scenario where the model serves as a multilingual agent that can process visual and textual inputs across languages in enterprise workflows. For example: a German engineering firm receives a maintenance request with attached photos from a Korean manufacturing partner, queries the model about the visual evidence in the photos, and receives an analysis that cross-references both German technical standards and Korean documentation—all in a single model without separate translation steps. The competitive IFEVAL score (89.6, exceeding Qwen3-VL-235B's 88.2) indicates reliable instruction-following, which is critical for agentic deployments where the model must execute multi-step procedures rather than answer single questions.