Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction

🎯 Pitch

Modern VLMs can read documents but still corrupt mathematical formulas and misorder text — a problem this survey reveals stems not from model capacity, but from naive end-to-end decoding that ignores page structure. The authors show modular pipelines decisively beat monolithic models on fine-grained accuracy benchmarks, making structure-aware decomposition the critical missing link.

1. Executive Summary

This survey systematically organizes document parsing research by proposing a taxonomy that divides the field into modular pipeline-based systems and unified Vision-Language Models (VLMs), reviewing approaches across layout analysis, OCR, mathematical expression recognition, table understanding, and visual element parsing. The paper tracks the evolution from task-specific pipelines toward end-to-end specialized VLMs—including multi-stage parsing (decoupled layout detection followed by region-level recognition, as in GLM-OCR and MinerU 2.5) and end-to-end parsing (direct image-to-markup generation within a single autoregressive model, as in DeepSeek-OCR and FireRed-OCR)—while consolidating evaluation metrics and benchmark datasets that span both subtask-level and holistic assessment protocols. The survey identifies verifier over-optimization and data scale as primary bottlenecks, establishing that unified VLM-based approaches outperform general-purpose VLMs on structure-sensitive metrics such as formula CDM and table TEDS only when paired with task-aligned training and controlled decomposition strategies, and that modular pipelines remain essential for fine-grained accuracy in complex real-world scenarios where end-to-end models exhibit reading-order inconsistencies and hallucination.

2. Context and Motivation

The Core Gap: No Unified Map of the Document Parsing Landscape

The fundamental problem this paper addresses is not the absence of document parsing techniques — the problem is that the field lacks a coherent organizational scaffold that connects isolated sub-tasks, modular pipelines, and the rapidly emerging paradigm of unified Vision-Language Models. Document parsing has evolved along two largely disconnected tracks. On one track, decades of research have produced mature, task-specific methods for layout analysis, OCR, formula recognition, and table extraction, each optimized within its narrow silo. On the other track, the past three years have seen an explosion of VLM-based systems that attempt to collapse the entire parsing workflow into a single model. The literature reflects this fragmentation: existing surveys either zoom in on one sub-task (layout analysis, OCR, table recognition) or provide broad overviews of document understanding that were written before VLMs reshaped the field. What is missing — and what this paper aims to provide — is a holistic taxonomy that treats document parsing as an integrated, page-level problem while tracking the methodological evolution from handcrafted pipelines to end-to-end multimodal generation.

This gap manifests concretely in three ways that the paper identifies (Section 2.2, Table 1):

First, task-specific surveys paint incomplete pictures. Surveys on document layout analysis — such as Binmakhashen and Mahmoud (2019) and Kumar and Lehal (2025) — cover rule-based and deep learning approaches to detecting text blocks, figures, and tables, but they do not extend to recognizing the content of those elements. Similarly, surveys on mathematical expression recognition (Aggarwal et al., 2022), table structure recognition (Kasem et al., 2024; Prajapati and Maurya, 2025), and chart understanding (Davila et al., 2020; Farahani et al., 2023) each provide deep dives into a single element type. These works are valuable for specialists, but they offer no guidance on how different recognition modules interact, how errors propagate between stages, or how the field is coalescing around unified architectures that handle all elements jointly.

Second, existing system-level surveys are outdated relative to the VLM revolution. Subramani et al. (2020) reviewed deep learning approaches for OCR and document understanding, and Arlazarov et al. (2022) covered document image analysis and recognition systems. Both surveys treat document understanding primarily as a combination of layout analysis and OCR, with downstream information extraction bolted on. Their taxonomies are insufficient to cover the full range of structured elements — mathematical expressions, chemical structures, complex charts — that modern documents contain and that modern systems must parse. More critically, these surveys were conducted before the rapid emergence of multimodal foundation models. They largely overlook the paradigm shift toward VLMs that enable end-to-end document parsing and structured generation from raw document images, a shift that began accelerating in 2023–2024 with models like Nougat, GOT, and the various specialized OCR-VLMs.

Third, the VLM era has produced a new set of design tensions that no survey has systematically analyzed. As VLMs have proliferated, the field has split into two technical trajectories — end-to-end generation (where a single model directly outputs Markdown/LaTeX/HTML from pixel inputs) and multi-stage decomposition within a unified model (where layout detection and content recognition are decoupled but share representations). These trajectories embody a fundamental architectural trade-off: end-to-end models promise global consistency and simplified system design, while multi-stage models offer efficiency, interpretability, and robustness to long documents. Practitioners face an increasingly confusing landscape of model choices — GOT-OCR2.0, DeepSeek-OCR, MonkeyOCR, MinerU, PaddleOCR-VL, GLM-OCR, SmolDocling, and many others — each with different design philosophies, parameter counts, and benchmark performance. The survey is the first to organize these emerging systems within a principled framework, distinguishing between general-purpose VLMs (Qwen2.5-VL, InternVL3, GPT-4o, Gemini), end-to-end specialized VLMs (DeepSeek-OCR, FireRed-OCR, dots.ocr, olmOCR), and multi-stage specialized VLMs (MonkeyOCR, MinerU 2.5, GLM-OCR, PaddleOCR-VL), while providing a consolidated performance comparison on OmniDocBench-v1.5 (Table 2).

Why the Gap Matters: Real-World and Theoretical Significance

Document parsing is not an academic exercise — it is the critical preprocessing bottleneck for an entire ecosystem of downstream applications that depend on converting unstructured documents into machine-readable representations. The paper highlights three application domains where parsing quality directly determines system performance (Section 1):

Retrieval-Augmented Generation (RAG). Modern RAG systems ingest PDFs, academic papers, financial reports, and legal documents, chunk them, embed them, and retrieve relevant passages for language models. If the document parser fails to extract tables properly, misidentifies reading order in multi-column layouts, or strips mathematical expressions into unparseable fragments, the retrieval quality degrades — and so does the final generation. Zhang et al. (2024) demonstrated this cascading effect explicitly in "OCR Hinders RAG," showing that OCR errors propagate through retrieval pipelines and degrade answer quality. The survey positions document parsing as the foundational input to RAG, meaning every improvement in parsing fidelity translates directly to better knowledge base construction and more reliable LLM-grounded responses.

Training data for large models. LLMs and multimodal foundation models are trained on vast corpora that increasingly include PDFs, scanned books, and web documents. The quality of structured extraction from these sources — particularly for scientific literature containing formulas, tables, and figures — determines what knowledge the model can access during pretraining. As the paper notes, structured document representations are central to building training corpora. A parser that correctly extracts LaTeX from a physics paper provides the model with semantically meaningful mathematical content; a parser that garbles it provides noise that degrades scientific reasoning capability.

Historical archives and industry digitization. Legal documents, medical records, government archives, and corporate filings exist largely in scanned or image-based formats. Parsing these at scale — with accurate layout preservation, table extraction, and formula recognition — unlocks knowledge that would otherwise remain trapped in unstructured pixels. The paper's emphasis on robustness to real-world distortions (blur, skew, warping, watermarks) reflects this practical reality: deployed systems face document quality far below the clean PDFs that dominate academic benchmarks.

Beyond these application drivers, the survey addresses a theoretical need for conceptual clarity. The vocabulary and organizing principles of document parsing are inconsistent across papers. Some works use "document understanding" as an umbrella term that includes question answering and information extraction; others use it synonymously with parsing. The term "OCR" has expanded from character recognition to encompass full-page structure extraction. The survey provides a formal problem definition (Section 2.1) that disambiguates these usages:

$f: \mathcal{D} \to \mathcal{S}$

where $\mathcal{D}$ is the space of document images and $\mathcal{S}$ is the space of structured representations (JSON, HTML, LaTeX, Markdown) that capture element types, spatial relationships, and reading order. This formalization clarifies that document parsing is distinct from both traditional OCR (which focuses on text transcription) and downstream document understanding tasks (which reason over parsed representations). The mapping $f$ must preserve not just content but structural organization — a constraint that distinguishes document parsing from generic image-to-text generation.

Where Prior Approaches Fall Short

The paper identifies specific limitations across both modular pipelines and early VLM-based systems that motivate its comprehensive review.

Modular pipeline fragility (Section 10.1). Pipeline-based systems decompose document parsing into sequential stages: layout detection → region cropping → element-specific recognition (OCR for text, specialized models for tables, formulas, charts) → structure assembly. This decomposition offers modularity and interpretability — each stage can be independently optimized and debugged — but it introduces two well-documented failure modes:

Error cascading: Minor inaccuracies in layout analysis propagate to subsequent stages without recovery. If a table is misdetected as a text block, the OCR module transcribes it as unstructured text rather than routing it to the table recognition module. If reading order is misidentified in a two-column paper, the output reads as interleaved gibberish. The paper notes that "the error tolerance of pipeline structures is relatively low" — a 2% layout detection error can cascade into far larger downstream accuracy losses because the error changes which recognition model processes which region.
Unified optimization is impossible: Each sub-module trains independently with its own objective function, making it "difficult to achieve end-to-end global optimization." The OCR model optimizes for character accuracy, the table model optimizes for structure recognition, and the layout model optimizes for bounding box recall — but there is no mechanism for them to jointly optimize for the final structured output. A table with slightly imperfect bounding boxes might be perfectly parseable by a more robust table recognizer, but the pipeline offers no way to communicate this slack between stages.

Despite these limitations, the paper emphasizes — importantly — that pipeline systems "remain essential for achieving robustness and fine-grained accuracy in complex real-world scenarios" (Section 11). This is not a "VLM replaces everything" narrative. The survey's balanced treatment is one of its distinguishing features: it argues that understanding pipeline components is necessary even for improving VLM-based systems, because VLM training data comes from pipeline-generated annotations and because hybrid architectures are re-introducing pipeline-style decomposition within unified models.

VLMs are powerful but unreliable for structure-sensitive tasks (Section 9.1). General-purpose VLMs like GPT-4o, Qwen2.5-VL, and InternVL3 achieve promising results on coarse-grained document tasks — layout recognition, reading-order reasoning, element-level question answering — but "when applied to fine-grained structured parsing, several intrinsic limitations become apparent." The paper identifies three specific weaknesses:

Hallucination in dense documents: VLMs tend to fabricate content or repeat passages when processing "dense, professionally typeset documents such as scientific PDFs." This is attributed to reliance on statistical priors from web-scale pretraining rather than explicit structural constraints. A VLM that has seen many papers "knows" that a references section typically follows the conclusion, and may generate reference entries that appear plausible but do not exist in the actual document.
Reading-order inconsistencies: Complex layouts — multi-column academic papers with floating figures and tables — violate the left-to-right, top-to-bottom assumptions baked into autoregressive generation. The paper cites this as a specific failure mode of end-to-end models, which "often manifest as reading-order inconsistencies in complex layouts." This is a direct consequence of collapsing a two-dimensional structure into a one-dimensional token sequence without explicit geometric reasoning.
Computational overhead: The quadratic complexity of self-attention over high-resolution visual tokens makes multi-billion-parameter VLMs impractical for industrial-scale document processing. A 72B-parameter model processing thousands of pages per hour is economically infeasible regardless of accuracy.

Table 2 in the paper quantifies these gaps: on OmniDocBench-v1.5, the best general-purpose VLM (Gemini-3.0 Pro) achieves 90.33 overall, but specialized models like FireRed-OCR (92.94) and GLM-OCR (94.62) substantially outperform it, particularly on structure-sensitive metrics like TableTEDS and Reading-Order Edit Distance, while using fewer parameters (2B and 0.9B respectively vs. unknown but certainly much larger for Gemini).

Existing benchmarks are fragmented and misaligned with real-world needs (Appendix I). Before OmniDocBench and similar holistic benchmarks, evaluation was conducted independently per sub-task: PubTabNet for tables, CROHME for formulas, PubLayNet for layout. This made it impossible to assess how errors in one component affect another, or to compare pipeline-based and end-to-end systems on equal footing. The paper highlights that "string-based similarity metrics remain prevalent due to their simplicity and universality" despite their failure to capture structural fidelity — a LaTeX expression that renders identically but uses different markup can receive a low edit-distance score, while a structurally garbled but lexically similar output can score high. Recent benchmarks like OmniDocBench and olmOCR-Bench address this by introducing structure-aware, unit-test-based, and attribute-based evaluation, but their adoption is still early and coverage remains incomplete relative to the "infinite variety of documents" (Section 10.2).

How This Paper Positions Itself

The paper does not claim to advance a new parsing method or achieve state-of-the-art results. Its contribution is taxonomic and organizational — it provides the framework that the field has been missing. The authors explicitly position their work as filling the gap between fragmented task-specific surveys and outdated system-level overviews (Section 2.2):

"These limitations highlight the need for a holistic and up-to-date survey that systematically organizes the field from a unified perspective."

The survey's positioning has several key dimensions:

It treats document parsing as a first-class problem, not an offshoot of OCR or document understanding. By providing a formal problem definition and a dedicated taxonomy, the paper establishes document parsing as a distinct research area with its own challenges, metrics, and design trade-offs. This contrasts with surveys that treat parsing as a preprocessing step within larger document understanding workflows.

It preserves the value of modular pipelines while embracing the VLM paradigm. Rather than declaring pipelines obsolete, the paper tracks how pipeline components are being absorbed into, combined with, and sometimes re-introduced within VLM architectures. The multi-stage vs. end-to-end distinction within specialized VLMs is a novel organizational contribution — prior work treated all VLM-based systems as a single category.

It provides the first consolidated performance comparison of VLM-based document parsers. Table 2 brings together general-purpose VLMs, end-to-end specialized models, and multi-stage specialized models on a single benchmark (OmniDocBench-v1.5), enabling direct comparison across design paradigms. This table is likely to become a reference point for the field, similar to how benchmark tables in NLP surveys serve as canonical performance snapshots.

It acknowledges limitations rather than presenting a settled narrative. The paper identifies open challenges including: the cost of difficulty estimation for compute-optimal strategies, the lack of VLM robustness to physical document distortions, the absence of combined search-and-revision approaches, and the fundamental unsolved problem of hard documents where no current method achieves meaningful accuracy. This honesty establishes the survey as a starting point for future work rather than a retrospective summary.

Its literature selection is systematic and transparent. The paper describes a structured search and screening process (Section 3.1) across six academic databases, using two-stage keyword queries (task-centric and technology-driven), with explicit inclusion/exclusion criteria. The final corpus of approximately 230 papers covers the period from roughly 1995 to 2026, spanning rule-based methods, statistical learning, deep neural architectures, and large-scale multimodal models. This methodological rigor distinguishes the survey from ad-hoc literature reviews and enables reproducibility.

In summary, the paper positions itself as the unifying reference that the document parsing community has been missing — a map of the terrain that connects historical pipeline research to contemporary VLM breakthroughs, organizes the proliferating model variants into a coherent taxonomy, and identifies the specific bottlenecks (verifier reliability, data scale, evaluation alignment) that must be addressed for the field to advance.

3. Technical Approach

3.1 Reader Orientation

This survey paper constructs a taxonomic framework — a systematic classification system — that organizes every major approach to document parsing into two fundamental architectural paradigms: modular pipeline-based systems and unified Vision-Language Models. The core problem is that the field lacks a shared conceptual vocabulary and organizational structure; the paper solves this by providing a formal problem definition, a hierarchical taxonomy grounded in architectural principles, a rigorous literature selection methodology, and a comprehensive map of how historical pipeline components relate to modern VLM architectures — giving researchers and practitioners a single reference point for understanding what methods exist, how they relate to each other, and where the field is heading.

3.2 Big-Picture Architecture (Diagram in Words)

The paper's technical apparatus consists of five interconnected components that collectively produce the comprehensive survey:

Literature Retrieval and Screening Pipeline — A systematic multi-stage process that searches six academic databases, applies keyword-based queries in two stages (task-centric and technology-driven), and filters retrieved papers through title/abstract screening and full-text eligibility assessment, yielding a final corpus of approximately 230 papers.
Taxonomic Classification Schema — A two-axis classification system whose primary axis distinguishes modular pipeline-based systems (explicit task decomposition into layout analysis, OCR, element recognition) from unified VLMs (end-to-end multimodal learning), and whose secondary axis tracks temporal evolution from rule-based methods through deep learning to multimodal foundation models.
Component-Level Survey Taxonomy — A hierarchical decomposition of modular pipelines into five functional modules (layout analysis, OCR, mathematical expressions, tables, visual elements), each further organized by architectural approach (CNN-based, Transformer-based, encoder-decoder, etc.) within a roughly chronological narrative.
VLM Taxonomy and Benchmarking Framework — A three-category classification of VLM-based parsers (general-purpose, end-to-end specialized, multi-stage specialized), grounded by a consolidated performance comparison table on the OmniDocBench-v1.5 benchmark.
Evaluation and Datasets Consolidation — A synthesis of widely adopted metrics (IoU-based detection metrics, edit-distance-based recognition metrics, structure-aware metrics like TEDS and CDM) and benchmark datasets, organized by sub-task and holistic evaluation settings.

Information flow: The literature retrieval produces a corpus → screened papers are organized into the taxonomic schema → each paper is analyzed within its architectural category, tracking methodological evolution → models are compared on shared benchmarks → evaluation protocols and datasets are cataloged as shared infrastructure. The taxonomy itself is the primary intellectual output; the literature review populates and validates it.

3.3 Roadmap for the Deep Dive

First, the formal problem definition of document parsing — because it establishes the mathematical framing that distinguishes document parsing from OCR and document understanding, enabling precise discussion of what methods must accomplish.
Second, the literature selection methodology — because the survey's validity depends on systematic, reproducible paper retrieval and filtering; this explains how the ~230-paper corpus was constructed and why certain works were included or excluded.
Third, the principles guiding taxonomy construction — because the classification system is not arbitrary; understanding the design criteria (architectural distinguishability, task generality, historical fidelity, alignment with evaluation practices) reveals why the taxonomy has the structure it does.
Fourth, the modular pipeline taxonomy — examining each functional component (layout analysis, OCR, mathematical expressions, tables, visual elements) in its architectural sub-categories, because these components are the building blocks that VLM-based systems either absorb, replace, or re-introduce.
Fifth, the VLM taxonomy — distinguishing general-purpose, end-to-end specialized, and multi-stage specialized models, because this is the survey's novel organizational contribution that no prior survey has captured.
Sixth, the evaluation metrics and benchmark consolidation — because metrics and benchmarks are the infrastructure that enables comparison across paradigms and drives methodological progress.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a taxonomic survey paper whose core idea is that document parsing methods can be systematically organized into two architectural paradigms — modular pipeline-based systems and unified Vision-Language Models — and that understanding the relationship between these paradigms, including their historical evolution and their complementary strengths and weaknesses, provides the coherent organizational scaffold the field has been missing.

Formal Problem Definition of Document Parsing

The paper defines document parsing as a structured transformation problem (Section 2.1). This formalization is essential because it establishes the mathematical scope of what document parsing systems must accomplish, distinguishing the task from both traditional OCR and downstream document understanding.

The core mapping is:

$f: \mathcal{D} \to \mathcal{S}$

where $\mathcal{D}$ is the space of document images (or rendered pages) and $\mathcal{S}$ is the space of structured document representations (such as JSON, HTML, LaTeX, or Markdown).

What it computes: Given a document page $D$ drawn from $\mathcal{D}$ , the function $f$ produces a structured output $S = f(D)$ that captures both the document's content and its structural organization — element types (text block, table, formula, figure), spatial relationships (bounding boxes, relative positions), and reading order (the logical sequence in which elements should be traversed).

Why this form: By defining document parsing as a mapping rather than a pipeline or a collection of tasks, the formalization accommodates both modular systems (where $f$ is implemented as a composition of sub-functions $f = f_n \circ \cdots \circ f_1$ ) and end-to-end systems (where $f$ is a single learned function). The output space $\mathcal{S}$ is deliberately underspecified — it can be Markdown, JSON, HTML, LaTeX, or any other structured format — because the survey's taxonomy must encompass all existing approaches without favoring a particular output convention. The formalization also clarifies the boundary between document parsing and document understanding: parsing produces the structured representation, while understanding tasks (question answering, information extraction, reasoning) operate on top of that representation.

A document $D$ is described as containing "heterogeneous elements, including text blocks, tables, mathematical expressions, charts, and other visual objects, arranged under a layout that encodes spatial and logical relationships." This characterization drives the modular pipeline decomposition: each element type requires specialized recognition capabilities, and the layout provides the spatial scaffolding that relates them.

Literature Selection Methodology

The survey's methodological contribution begins with its literature retrieval and screening protocol (Section 3.1), which follows a structured process modeled on systematic review methodologies in computing research. This protocol is important because it makes the survey's paper selection reproducible and transparent — unlike ad-hoc literature reviews, the reader can assess whether the corpus is comprehensive and unbiased.

Search Scope and Databases (Section 3.1.1). The paper searches across six sources:

ACM Digital Library and IEEE Xplore — which index major venues in document analysis and computer vision, providing coverage of the core document analysis community.
Web of Science and Scopus — which capture interdisciplinary publications across information retrieval and machine learning, ensuring coverage of works published outside dedicated document analysis venues.
Google Scholar — which provides broad cross-domain coverage and captures preprints and gray literature that may not appear in curated databases.
arXiv — which provides access to recent advances in multimodal and vision-language modeling, a rapidly moving area where peer-reviewed publication lags behind technical progress.

Together, these sources cover research across document analysis, computer vision, natural language processing, and multimodal learning — the four disciplines that contribute to modern document parsing.

Search Keywords and Query Design (Section 3.1.2). The search strategy is organized into two stages that capture both task-centric and technology-driven perspectives:

First stage (task-centric keywords, no methodology restriction): "document parsing," "layout analysis," "OCR," "table recognition," "formula recognition," "chart parsing," "OCSR" (Optical Chemical Structure Recognition), and "document structure extraction." These terms correspond to the fundamental sub-tasks of document parsing and ensure coverage of classical approaches including rule-based and early machine learning methods. By not restricting methodology, this stage prevents the survey from being biased toward deep learning approaches.

Second stage (technology-driven keywords): "encoder-decoder document analysis," "vision-language model for document parsing," and "multimodal document understanding." These terms target recent advances driven by Transformer architectures, large-scale pretraining, and the integration of vision and language models. This stage ensures that the survey captures the VLM paradigm shift that prior surveys missed.

Across databases, queries are formulated using Boolean combinations of task and method terms. The search covers publications from approximately 1995 to 2026, capturing both early foundational work (rule-based layout analysis, classical OCR) and recent deep learning-based advances.

Screening and Eligibility (Section 3.1.3). The retrieved records pass through a multi-stage filtering process, illustrated in Figure 2:

Identification: The initial search yields approximately $N \approx 860$ records.
Deduplication: Removing duplicate entries across databases reduces the pool to approximately $N \approx 680$ .
Title and abstract screening: This stage excludes "irrelevant studies and application-focused work without methodological contributions," leaving approximately $N \approx 429$ papers. The key criterion here is whether the paper proposes or evaluates a method — pure application papers that use existing tools without methodological novelty are excluded.
Full-text eligibility assessment: The remaining papers are assessed for "methodological relevance and experimental rigor." Studies are included if they (a) propose or evaluate methods for document parsing or closely related sub-tasks and (b) provide sufficient technical detail and empirical validation. Works are excluded if they lack experimental evaluation, contain only high-level system descriptions without technical depth, or focus solely on application deployment without methodological contribution.
Final inclusion: After this process, approximately $N = 230$ papers are retained for detailed analysis.

This multi-stage filtering produces a corpus that is both comprehensive (covering the full historical and methodological range) and focused (excluding work without methodological contribution or empirical validation). The explicit inclusion/exclusion criteria enable reproducibility and allow readers to assess potential coverage gaps.

Principles for Taxonomy Construction

The paper does not impose an arbitrary classification; it derives its taxonomy from four design principles (Section 3.2) that ensure the resulting framework is conceptually coherent, applicable across heterogeneous tasks, historically grounded, and aligned with evaluation practices.

Principle 1: Architectural distinguishability. Methods should be grouped according to their underlying system organization rather than superficial implementation differences. In document parsing, the paper identifies architectural design as the primary determinant of "how visual features, textual content, and structural representations are integrated." This principle motivates the top-level split between modular pipelines (where each component has a separate architecture optimized for its specific task) and unified VLMs (where a single architecture handles all tasks through shared representations). Within each paradigm, further architectural distinctions — CNN vs. Transformer, detection-based vs. segmentation-based, encoder-decoder vs. autoregressive — provide the secondary classification axes.

Principle 2: Task generality. The taxonomy must apply across the heterogeneous sub-problems that constitute document parsing — layout analysis, OCR, table understanding, mathematical expression recognition, and chart interpretation — "rather than being tailored to a specific application." This principle ensures that the taxonomy is not, for example, a classification of table recognition methods that happens to mention other tasks, but a genuine cross-cutting framework. The paper achieves this by organizing the modular pipeline section by task (Section 4-8) and the VLM section by architectural paradigm (Section 9), with each task section internally organized by the same architectural distinctions (CNN-based, Transformer-based, etc.).

Principle 3: Historical fidelity. The taxonomy should "reflect the historical evolution of the field" from rule-based and heuristic systems through statistical learning pipelines to unified multimodal architectures. This is implemented through the roughly chronological organization within each component section. For example, the layout analysis section (Section 4) progresses from CNN-based methods (which dominated roughly 2015-2020) to Transformer-based methods (2020-present) to multimodal approaches integrating LLMs (2023-present). This chronological structure helps contextualize current approaches by showing "how modeling assumptions have shifted over time."

Principle 4: Alignment with evaluation practices. The classification should "align with common evaluation practices, datasets, and supervision strategies." Methods that share similar training objectives, supervision signals, and evaluation protocols often exhibit consistent design patterns and can be naturally grouped together. For example, table detection methods are grouped by their underlying object detection framework (Faster R-CNN, YOLO, DETR) because these frameworks share evaluation protocols (mAP at specific IoU thresholds). This principle makes the taxonomy practically useful: a practitioner evaluating table detection methods can locate all Faster R-CNN-based approaches in one place and compare their reported metrics directly.

The Architectural Classification Axis

Guided by these principles, the survey's primary taxonomic axis distinguishes between two architectural paradigms (Section 3.3):

Modular pipeline-based systems follow an explicit task decomposition strategy. Document parsing is divided into a sequence of intermediate sub-tasks — layout detection, optical character recognition, table structure extraction, mathematical expression parsing, visual element analysis — where each component is optimized separately with task-specific supervision and objective functions. Intermediate representations (bounding boxes, OCR tokens, structural graphs) are propagated between stages. The paper characterizes this paradigm as emphasizing "modularity, interpretability, and controllable optimization," with the trade-off that it "requires carefully designed task boundaries and coordination across heterogeneous modules." The formalism is:

$f = f_n \circ f_{n-1} \circ \cdots \circ f_1$

where each $f_i$ is a task-specific function (e.g., $f_1$ for layout detection, $f_2$ for OCR, $f_3$ for table recognition) and the composition propagates intermediate structured representations between stages.

Unified Vision-Language Models approach document parsing as an end-to-end multimodal learning problem. Rather than decomposing the problem into explicitly defined intermediate stages, these models "learn shared visual and textual representations and directly generate structured outputs." The typical implementation uses large Transformer-based architectures that benefit from "large-scale multimodal pretraining and parameter sharing, enabling stronger cross-task generalization." The paper identifies this paradigm as reflecting "a broader shift in artificial intelligence from staged processing pipelines toward unified representation learning and generative modeling."

Within the unified VLM paradigm, the paper introduces a further sub-classification that distinguishes between:

End-to-end specialized VLMs: Models that perform fully end-to-end structured generation without explicit intermediate supervision — a single autoregressive model directly converts raw page images into structured representations like Markdown or LaTeX. Examples include DeepSeek-OCR, GOT-OCR2.0, and FireRed-OCR.
Multi-stage specialized VLMs: Models that retain certain multi-stage reasoning or intermediate decoding strategies within a unified parameter space — they decompose the process into interpretable sub-tasks (e.g., layout detection followed by region-level recognition) but use neural rather than heuristic interfaces between stages. Examples include MonkeyOCR, MinerU 2.5, and GLM-OCR.

Why this distinction matters: The paper argues that these two sub-paradigms reflect "different design choices regarding structural decomposition and optimization granularity under the unified modeling paradigm." End-to-end models optimize for global consistency by collapsing layout, recognition, and structure inference into a single sequence generation task. Multi-stage models reintroduce controlled decomposition to improve efficiency (avoiding full-page quadratic attention), reduce hallucination (by constraining recognition to pre-detected regions), and handle long documents (by processing regions independently). The taxonomy captures both without declaring one superior, instead identifying their complementary trade-offs.

The Modular Pipeline Taxonomy: Layout Analysis

Section 4 organizes document layout analysis (DLA) — the task of detecting and classifying structural elements like text blocks, tables, figures, and formulas while determining their spatial relationships — into three research directions that have developed since 2020.

Visual-based Layout Detection (Section 4.1). This direction treats document pages as images and detects layout elements using visual features, primarily through object detection and segmentation models.

CNN-based Methods (Section 4.1.1): These were "among the earliest deep learning models applied to document layout analysis." Object detection frameworks such as Faster R-CNN and Mask R-CNN are adapted to detect layout elements including text blocks, tables, and figures. The paper notes that these methods "significantly improved detection accuracy compared to rule-based approaches by learning hierarchical visual representations directly from data." However, their "locality-biased feature extraction can make them less effective when long-range structural dependencies are critical." Specific recent instantiations include:

DocLayout-YOLO: Introduces a "Global-to-Local Controllable Receptive Module to improve multi-scale layout detection while maintaining fast inference speed." This addresses the challenge that document elements vary enormously in scale — from full-page figures to small footnote markers.
YOLO-DLA: Addresses multi-scale detection where "macro-scale elements (e.g., text, tables, figures) coexist with micro-scale elements such as headings, captions, and formulas." It introduces Kernel Weighting Convolution and a "scale-aware curriculum learning strategy that progressively trains models from macro to micro elements," significantly improving detection of small layout components often overlooked by previous models.
PP-DocLayout: A unified framework supporting 23 layout categories across diverse document types (academic papers, books, magazines, exam papers) that "achieves real-time inference speeds exceeding 120 pages per second," demonstrating feasibility for large-scale document data construction.

Transformer-based Methods (Section 4.1.2): These address the limitation that CNNs "struggle to model long-range dependencies across document regions" by using self-attention mechanisms. Vision Transformer-based models like BEiT and the Document Image Transformer (DiT) "apply patch-based representations to document images, enabling models to learn global layout patterns." The trade-off is that "the quadratic complexity of self-attention makes them computationally expensive for high-resolution document images." Recent work explores efficient variants: DocSemi proposes an additive-attention-based semi-supervised framework that "integrates this efficient attention mechanism into a student–teacher training pipeline, enabling accurate layout detection with reduced computational overhead and limited annotated data."

Multimodal Layout Understanding (Section 4.2). This direction integrates textual and semantic information with visual features, recognizing that "understanding document structure often requires integrating textual and semantic information." The paper traces the evolution through:

LayoutLM family (LayoutLM → LayoutLMv2 → LayoutLMv3): These models "integrate textual content and layout information within a unified Transformer architecture" by combining word embeddings with positional and visual features. LayoutLMv2 and v3 "further enhanced multimodal interactions through improved cross-modal pretraining objectives and masking strategies."
Alternative representations: Grid-based approaches (CharGrid, BERTGrid) "represent documents as structured grids that preserve spatial relationships between characters and tokens." Graph-based models "treat document components as nodes in a graph and model structural relationships between them," improving the ability to capture hierarchical and relational layout structures.
LLM integration: LayTokenLLM "proposes representing layout information as lightweight tokens that are interleaved with textual content and fed into LLMs," using "a single layout token per text segment and a specialized positional encoding scheme." This enables LLMs to reason about document structure for tasks like question answering. ColParse "utilizes document parsing models to generate a compact set of layout-informed sub-image embeddings that are fused with global page representations, enabling efficient and structurally-aware visual document retrieval."

Emerging Trends (Section 4.3). Two challenges receive particular attention:

Dataset diversity: M6Doc introduces "a large-scale dataset containing multiple document formats (PDF, scanned, and photographed documents), multiple document types, and bilingual annotations," with "74 fine-grained layout categories and more than 200,000 annotations," providing a comprehensive benchmark for fine-grained logical layout analysis.
Semi-supervised learning: DocSemi proposes "a DETR-based semi-supervised framework that combines one-to-one and one-to-many assignment strategies through a hybrid matching mechanism," using focused attention networks and guided query strategies within a teacher-student framework to generate high-quality pseudo-labels with limited labeled data.

The Modular Pipeline Taxonomy: Optical Character Recognition

Section 5 organizes OCR into three technical paradigms — text detection, text recognition, and unified text spotting — while tracking the transition toward multimodal intelligence.

Foundations and Classical Pipelines (Section 5.1). The classical OCR pipeline separates detection and recognition:

Text detection methods include regression-based approaches like TextBoxes (which "directly predict bounding boxes") and EAST, alongside segmentation-based methods like PixelLink (which "handles irregular shapes by classifying pixels or character regions") and CRAFT (which detects character regions). Segmentation-based methods are characterized as "usually more robust than regression-based detectors to curved or irregular text" but "often require more elaborate post-processing to recover coherent text instances."
Text recognition models include CTC-based architectures exemplified by CRNN (which handles "sequence alignment without explicit segmentation") and attention-based Sequence-to-Sequence models that "integrated visual features with linguistic context, significantly improving accuracy on distorted or blurred text." Transformer-based architectures like TrOCR further advanced this integration.
Text spotting unifies detection and recognition through shared feature representations, with frameworks like ABCNet (which uses adaptive Bezier-curve fitting for irregular text) and SwinTextSpotter. These are characterized as improving "consistency between localization and transcription relative to modular pipelines" while making "diagnosis and correction of failure cases less straightforward."

Modern Research Frontiers (Section 5.3). The paper identifies three active frontiers where independent OCR research remains vital despite the shift toward general VLMs:

Vertical and complex scenarios: Specialized domains where general VLMs may falter. DeepAd-OCR "leverages AI-enhanced OCR to optimize conversion elements in digital advertisements in real-time, integrating deep reinforcement learning to balance recognition accuracy with business metrics like conversion rates and regulatory compliance."
Internal mechanism and interpretability: Research into "OCR Heads" identifies "specialized attention units within LVLMs that are distinct from standard text-retrieval heads," which "focus on visual patches to guide text extraction, offering a mechanistic path to reduce hallucinations and improve grounding in multimodal reasoning."
Uncertainty quantification: The Consensus Entropy (CE) framework "introduces a novel uncertainty metric based on inter-model agreement" by "calculating the semantic divergence among multiple VLM predictions," enabling "adaptive routing — merging high-confidence results while redirecting high-entropy cases to more powerful specialized models for secondary verification."

The Modular Pipeline Taxonomy: Mathematical Expression Detection and Recognition

Section 6 organizes mathematical expression processing into detection and recognition stages, with the paper noting that detection is increasingly absorbed into general layout analysis while recognition remains an active research area.

Mathematical Expression Detection (Section 6.1). Early methods treated expressions as visual objects detected via CNN or handcrafted features. Later approaches adapted general object detection frameworks (SSD, YOLO, Faster R-CNN, Mask R-CNN). More recent context-aware methods like FormulaDet "formulate mathematical expression detection as an entity–relation extraction problem and leverage layout-aware modeling to improve robustness." The paper notes that with modern unified frameworks, "mathematical expression detection is rarely treated as an independent task" — it is typically handled by document layout analysis models that detect expressions alongside other elements.

Mathematical Expression Recognition (Section 6.2). This task — converting an expression image into structured markup like LaTeX — is significantly more challenging than standard OCR because models "must not only recognize individual symbols but also infer their spatial and hierarchical relationships."

Encoder-Decoder Models and Sequence-based Recognition (Section 6.2.1): Most modern systems adopt encoder-decoder architectures. BTTR "employs a Transformer decoder for handwritten mathematical expression recognition," while CoMER "improves symbol alignment and decoding accuracy through refined attention mechanisms." The paper notes a fundamental limitation: these models "still linearize inherently two-dimensional structures, which limits their ability to represent hierarchical relations explicitly."

Structure-aware Modeling (Section 6.2.2): This line of work incorporates explicit structural modeling:

TAMER: Introduces "a tree-aware Transformer architecture that jointly learns sequence prediction and expression tree structures," with a Tree-Aware Module that "enhances the model's ability to capture hierarchical relationships while maintaining efficient training."
SSAN: Introduces "a symbol spatial-aware network that predicts spatial distribution maps for symbols as an auxiliary task, allowing the model to better capture two-dimensional layouts."
SCAN: Proposes "a Skeleton Shaping and Character Counting Module that simultaneously predicts expression skeleton structures and symbol frequency distributions, enabling the model to correct potential symbol misrecognition caused by visually similar characters."
TAN: Incorporates "symbol-type labels into an attention-based encoder–decoder framework," providing explicit supervision on symbol categories to improve feature representation for visually similar handwritten characters.

Visual Representation and Scaling (Section 6.2.3): AutoScaler addresses the "scale misalignment problem" — mathematical expressions exhibit large variations in scale and aspect ratio that cause information loss with fixed-resolution inputs — by proposing "a self-scale alignment framework that adaptively determines the optimal input scale for each mathematical expression image" through stochastic scale training and scale-aware inference.

Data and Efficiency Considerations (Section 6.2.4): The MathWriting dataset introduces "more than 600k handwritten mathematical expressions, providing one of the largest publicly available datasets." PP-FormulaNet constructs "a large-scale mathematical expression corpus from arXiv papers and proposes a lightweight recognition framework combining knowledge distillation and multi-token prediction." UniRec-0.1B proposes "a compact unified recognition model capable of handling both text and mathematical expressions" with only 0.1B parameters, supported by "hierarchical supervision and a semantic-decoupled tokenizer."

VLM-based Recognition (Section 6.2.5): Recent work applies multimodal large models to mathematical expression recognition:

Uni-MuMER proposes "a unified multi-task fine-tuning framework that enhances open-source VLMs for handwritten mathematical expression recognition" by integrating auxiliary tasks such as symbol counting and tree-aware reasoning.
HiE-VL introduces "a hierarchical adapter architecture consisting of primitive-level and structural adapters to better capture fine-grained visual features and hierarchical relationships within mathematical expression images, together with a progressive training strategy."

The Modular Pipeline Taxonomy: Table Detection and Recognition

Section 7 organizes table processing into detection and structure recognition, with the paper noting that detection has "achieved relatively mature performance on many benchmarks" and research efforts have shifted toward the more challenging structure recognition tasks.

Table Detection (Section 7.1). Detection is approached as an object detection task where tables are treated as objects. The paper surveys adaptations of standard detection frameworks:

PDF feature integration: Methods that integrate PDF-specific features like character coordinates into CNN-based models (Hao et al., 2016).
Faster R-CNN adaptations: Customizations including "modifying representation and optimizing anchor points" for document images (Gilani et al., 2017) and combining Deformable CNNs to handle varying table scales (Schreiber et al., 2017).
YOLO-based methods: Adaptations that "enhance anchor and post-processing techniques" (Huang et al., 2019).
Sparse detection: An approach that "expands SparseR-CNN with Gaussian Noise Augmented Image Size proposals and many-to-one label assignments, introducing the Information Coverage Score (ICS) to evaluate recognition accuracy" (Xiao et al., 2023), which is "particularly useful for sparse or weakly bounded tables where standard detectors tend to miss large empty regions."

Table Structure Recognition (Section 7.2). The paper organizes recognition methods into three architectural approaches:

Row and Column Segmentation (Section 7.2.1): These top-down methods first identify the overall table region and then segment it into rows and columns.

Row and column detection: Initial approaches used object detection algorithms for table bounding boxes, then segmentation algorithms to establish row-column relationships. Transformers like DETR "excel at recognizing global relationships within an image." Specific innovations include row and column segmentation through transformer queries (TRUST) and dynamic query enhancement (DQ-DETR). Bi-GRUs "effectively captured row and column separators by scanning images bidirectionally" (Khan et al., 2019).
Fusion modules: Advanced algorithms "estimate merging probabilities between cells to improve recognition accuracy in tables without explicit row and column lines." Techniques include embedding modules that "integrate plain text within grid contexts to guide merge predictions via GRU decoders" and "adjacency criteria and spatial compatibility to predict cell mergers."

Cell-based Methods (Section 7.2.2): These bottom-up approaches "construct table structures by first detecting individual cells and then modeling the relationships between them."

Cell detection improvements: HRNet-based backbones for high-resolution feature extraction (CascadeTabNet), specialized loss functions including continuity and overlap constraints, and dual-path architectures that jointly learn local features and segmentation masks.
Geometric representations: Vertex prediction methods detect cell corners to handle perspective distortions (Cycle-Pairing Module). Graph-based approaches "represent tables as graphs, where cells are nodes and spatial relationships define edges, enabling Graph Neural Networks (GNNs) to model complex adjacency relationships" (Chi et al., 2019; Qasim et al., 2019).

Image-to-Sequence Approaches (Section 7.2.3): These methods "reformulate table recognition as a sequence generation task, converting table images directly into structured representations such as LaTeX or HTML."

Encoder-decoder models: Early studies explored converting scientific table images into LaTeX code (Deng et al., 2019). Later works introduced dual-decoder architectures "to handle structural tags and textual content simultaneously" (Zhong et al., 2020). MASTER "further improved sequence modeling capabilities for complex table layouts."
Transformer-based architectures: TransTab "introduces Vision Transformers (ViT) to model long-range dependencies within table layouts, improving the detection of complex row and column relationships."
End-to-end alignment: E2eTRNet "introduces a dual-decoder architecture where semantic features associated with structure tokens guide the prediction of cell content, enabling automatic alignment and eliminating the need for explicit bounding-box matching."
Unified frameworks: UniTable "reformulates table recognition as an image-to-text task, jointly predicting table structure, cell content, and bounding boxes using a unified sequence generation objective," leveraging "self-supervised pretraining on large-scale unlabeled table images."
VLM-based approaches: TRivia "proposes a self-supervised fine-tuning framework that enables VLMs to learn table recognition from large collections of unlabeled table images" through "a question-answering-based proxy task and reinforcement learning optimization to generate supervisory signals automatically."

The Modular Pipeline Taxonomy: Visual Element Parsing

Section 8 covers two specialized non-textual elements: charts and chemical structures.

Chart Parsing (Section 8.1). The paper traces the evolution from pipeline-based systems to end-to-end multimodal generation:

Pipeline-based methods: Traditional approaches involve chart classification, element detection, text recognition through OCR, and heuristic rule-based reasoning to associate visual elements with textual labels. These methods "achieved reasonable performance for simple charts" but "suffered from error propagation and limited robustness across diverse chart styles and layouts."
End-to-end multimodal generation: ChartAssistant "introduces a chart-to-table pre-training task that converts charts into Markdown tables, enabling the model to learn structural relationships between graphical elements and numerical values before multitask instruction tuning." TinyChart "improves chart understanding by reducing redundant visual tokens through a visual token merging strategy and enhancing numerical reasoning via Program-of-Thoughts learning, where the model generates executable Python programs to perform intermediate calculations." ChartMoE "explores richer structured representations by aligning charts with multiple modalities, including tables, JSON attributes, and visualization code, using a mixture-of-experts connector."
Chart-to-code generation: VinciCoder "explores multimodal code generation models that directly produce visualization code from chart images and improves visual fidelity through a visual reinforcement learning framework that optimizes generated programs based on the similarity between rendered and target charts." The Chart2Code benchmark "introduces a hierarchical evaluation framework with three progressively challenging tasks — chart reproduction, chart editing, and long-table-to-chart generation."

Optical Chemical Structure Recognition (Section 8.2). This specialized task converts graphical depictions of chemical molecules into machine-readable formats like SMILES or InChI.

Rule-based methods: Tools like OSRA and chemoCR "relied on binarization, skeletonization, and heuristic rules to assemble the molecular graph" but were "highly sensitive to image artifacts, such as broken lines or overlapping text."
Deep sequence generation: SwinOCSR "employs a Swin Transformer as a hierarchical vision backbone to capture multi-scale structural features, converting them into DeepSMILES strings." MPOCSR "introduces a multi-path Vision Transformer (MPViT) and a class-balanced loss function to mitigate the long-tail distribution of chemical elements."
MLLM and semantic optimization: MolParser "addresses the challenge that standard SMILES often fails to represent specialized entities like Markush structures in patents" by introducing "an extended SMILES (E-SMILES) format and a large-scale dataset (MolParser-7M)." MolSight "introduces a reinforcement learning framework using the GRPO algorithm, which optimizes the model based on chemical semantic correctness rather than just token-level accuracy, allowing for superior recognition of challenging stereoisomers."

The VLM Taxonomy

Section 9 organizes VLM-based document parsing into three paradigms, culminating in a consolidated performance comparison on OmniDocBench-v1.5 (Table 2).

General-Purpose VLMs (Section 9.1). These models "were not originally designed for structured document parsing; however, they have played a foundational role in advancing multimodal document understanding." The paper surveys models including Qwen2.5-VL (72B), InternVL3 (76B, 241B), Qwen3-VL (2B, 235B), GPT-4o, GPT-5.2, Gemini 2.5/3 Pro, Kimi-VL, and Claude Sonnet 3/4. Their strengths lie in "scale, broad pretraining, and strong instruction-following capabilities, which enable flexible reasoning over heterogeneous document types without task-specific engineering." However, when applied to fine-grained structured parsing, they exhibit:

Hallucination and repetitive generation in "dense, professionally typeset documents such as scientific PDFs."
Reading-order inconsistencies because "their training objectives emphasize broad visual grounding and conversational alignment rather than hierarchical structural modeling."
Computational overhead due to "multi-billion-parameter models and quadratic attention over high-resolution visual tokens."

End-to-End Specialized VLMs (Section 9.2). These models "aim to directly convert raw page images into structured representations such as Markdown, LaTeX, or HTML" by "jointly modeling layout detection, content recognition, and structural relation inference within a unified neural architecture." The paper's key insight is that "structural coherence is best preserved when layout geometry and textual decoding are optimized simultaneously rather than sequentially."

The paper traces the evolution:

Early dedicated VLMs: Nougat and mPLUG-DocOwl "demonstrated the feasibility of direct image-to-markup generation." Vary "introduced specialized vision vocabularies to better align dense document regions with structured outputs."
GOT-OCR2.0: Extended the idea through "General OCR Theory" (OCR-2.0), advocating "a unified decoding framework for diverse artificial optical signals — including text, formulas, tables, charts, and even sheet music." It combines "a high-compression encoder with a long-context decoder" for "high-precision full-page parsing at substantially lower inference cost than general multi-billion-parameter VLMs."
Efficiency-focused models: SmolDocling "demonstrated that a compact 256M-parameter model can remain competitive when paired with an appropriate lightweight structured format." UniRec-0.1B "explored extreme model efficiency by proposing a unified text-and-formula recognition model with only 0.1B parameters." dots.ocr "strengthened multilingual robustness by jointly modeling layout and relational structures across 126 languages."
Reinforcement learning integration: Logics-Parsing "introduced a two-stage SFT-then-RL strategy with layout-aware rewards to enforce natural reading sequences." Infinity-Parser "formalized document parsing as a LayoutRL problem, optimizing hierarchical consistency through document-level rewards." olmOCR 2.0 "introduced Reinforcement Learning with Verifiable Rewards (RLVR), employing binary unit tests to validate mathematical expressions and table structures." OCRVerse "proposes a holistic end-to-end framework that unifies traditional text-centric OCR with vision-centric document understanding tasks, combining cross-domain SFT with domain-specific reinforcement learning rewards."
Architectural innovations: DeepSeek-OCR "reframed the vision-language interface from an LLM-centric perspective, treating visual tokens as an efficient compression medium for textual information and achieving 7-20× effective compression." DeepSeek-OCR 2 "introduced DeepEncoder V2, replacing CLIP-style encoders with a compact LLM-based architecture that models a 'causal visual flow' — reordering visual features according to semantic dependencies rather than fixed raster order."
Large-scale instantiation: HunyuanOCR "represents a recent large-scale instantiation of the end-to-end paradigm" with "1B parameters, 200M application-aligned pretraining samples, and online reinforcement learning via GRPO." FireRed-OCR "introduces a progressive training framework that converts a general-purpose VLM into a pixel-precise structural OCR system through geometry-aware data generation and a three-stage training curriculum combining multi-task alignment, structured SFT, and format-constrained GRPO optimization."

Multi-Stage Specialized VLMs (Section 9.3). These architectures "reintroduce controlled decomposition while retaining neural integration," motivated by "the scalability and robustness limitations of unified generation." Rather than collapsing everything into a single sequence, they "modularize the process into interpretable sub-tasks, thereby improving efficiency and reducing hallucination in high-resolution, text-dense scenarios."

MonkeyOCR: Introduces the "Structure-Recognition-Relation (SRR) paradigm, explicitly separating layout detection, content recognition, and relational modeling." It "avoids full-page quadratic self-attention" and "enables block-level parallel recognition while preserving fine-grained details." MonkeyOCR v1.5 "further simplifies the pipeline and incorporates reinforcement learning based on visual consistency, alongside Image-Decoupled Table Parsing (IDTP) for complex and cross-page tables."
MinerU 2.5: Proposes "a decoupled coarse-to-fine inference mechanism, first performing global layout analysis on downsampled images and subsequently applying high-resolution recognition to cropped regions." This "reduces token redundancy and significantly mitigates hallucination, enabling a 1.2B-parameter model to outperform much larger general-purpose VLMs in long-document processing."
GLM-OCR: "Exemplifies a compact yet high-performance multi-stage system, integrating a 0.4B-parameter CogViT visual encoder with a 0.5B-parameter GLM language decoder." It uses "a two-stage pipeline — PP-DocLayout-V3 for layout analysis followed by parallel region-level recognition — and a Multi-Token Prediction mechanism to accelerate decoding."
PaddleOCR-VL: Emphasizes robustness to real-world distortions. Its PP-DocLayoutV3 "replaces rectangular bounding boxes with multi-point representations to model non-planar distortions such as warping and skewing."

Table 2: Consolidated Performance on OmniDocBench-v1.5. The paper provides a comprehensive performance comparison that reveals several patterns:

General-purpose VLMs: Gemini-3.0 Pro achieves 90.33 overall (best among general models), with TextEdit of 0.065, FormulaCDM of 89.18, TableTEDS of 88.28, and TableTEDS_s of 90.29. GPT-5.2 achieves 85.50 overall. Qwen3.5 (397B) achieves 90.80 overall.
End-to-end specialized VLMs: FireRed-OCR (2B) achieves 92.94 overall — substantially outperforming all general-purpose VLMs despite its smaller size — with TextEdit of 0.032, FormulaCDM of 91.71, TableTEDS of 90.31, and TableTEDS_s of 93.81. DeepSeek-OCR 2 (3B) achieves 91.09 overall.
Multi-stage specialized VLMs: GLM-OCR (0.9B) achieves 94.62 overall — the highest reported — with TextEdit of 0.030, FormulaCDM of 93.90, TableTEDS of 93.96, and TableTEDS_s of 96.39. PaddleOCR-VL-1.5 (0.9B) achieves 94.50 overall.

The pattern is that multi-stage specialized VLMs achieve the highest overall scores while using the smallest parameter counts, outperforming both general-purpose VLMs and end-to-end specialized VLMs on structure-sensitive metrics, particularly table structure recognition and reading order.

Evaluation Metrics and Benchmarks (Appendix I)

The paper consolidates evaluation protocols across sub-tasks, organized by the type of output being evaluated.

Layout Analysis Metrics (Appendix I.1). The core metric is Intersection over Union:

$\text{IoU} = \frac{\text{Area of Overlap}}{\text{Area of Union}}$

where Area of Overlap is the intersection of predicted and ground-truth regions and Area of Union is their union.

What it computes: The ratio of overlapping area to total area spanned by two bounding boxes, yielding a value in $[0, 1]$ where 1 is perfect overlap and 0 is no overlap. Based on IoU thresholding, Precision, Recall, and F1-score are computed.

Why this form: IoU normalizes for scale — a 5-pixel offset matters proportionally the same on a small text block and a large figure — and is the standard metric in object detection, enabling direct comparison with methods from the broader computer vision literature.

For aggregated evaluation, mean Average Precision is:

$\text{mAP} = \frac{1}{N}\sum_{i=1}^{N} \text{AP}_i$

where $N$ is the number of element categories and $\text{AP}_i$ is the average precision for category $i$ .

What it computes: The mean of per-class average precision scores, each computed as the area under the precision-recall curve at varying confidence thresholds. mAP@IoU[a:b] averages mAP over a range of IoU thresholds (e.g., 0.5-0.95), providing a stricter assessment of localization quality.

Why this form: Averaging across categories prevents dominant classes from masking poor performance on rare elements, and averaging across IoU thresholds penalizes methods that produce approximately correct but imprecisely localized boxes.

The paper also introduces Page-IoU (proposed in MinerU 2.5), a page-level coverage metric:

$\text{PageIoU}(P, G) = \frac{\sum_{p \in M} \min\{P_{\text{cover}}(p), G_{\text{cover}}(p)\}}{\sum_{p \in M} \max\{P_{\text{cover}}(p), G_{\text{cover}}(p)\}}$

where $M$ is the non-background region of the page, $P_{\text{cover}}(p)$ is the number of predicted elements covering pixel $p$ , and $G_{\text{cover}}(p)$ is the number of ground-truth elements covering pixel $p$ .

What it computes: For each pixel in the non-background region, it computes the minimum and maximum of predicted and ground-truth coverage counts. The sum of minima (agreement) is divided by the sum of maxima (total extent), yielding a measure of pixel-level layout consistency.

Why this form: Unlike box-matching IoU, Page-IoU does not require one-to-one correspondence between predicted and ground-truth boxes. This makes it robust to annotation granularity differences — a large text block prediction that overlaps several ground-truth text lines receives partial credit proportional to its pixel-level overlap rather than being penalized for not matching any single box exactly.

Text Recognition Metrics (Appendix I.1). The fundamental metric is Edit Distance (Levenshtein distance), which measures the minimum number of insertions, deletions, and substitutions to transform a predicted string into the ground truth. Normalized forms include:

Character Error Rate (CER): Edit distance normalized by the length of the ground truth string, measuring per-character accuracy.
Word Error Rate (WER): Edit distance computed at the word level rather than character level, more sensitive to semantically meaningful errors.

Additionally, BLEU measures n-gram overlap between prediction and reference, providing partial credit for partially correct sequences. METEOR incorporates both precision and recall with flexible matching through stemming and synonym alignment.

Mathematical Expression Recognition Metrics (Appendix I.1). The core metric is Character Detection Matching (CDM):

$\text{CDM} = \frac{2\text{TP}}{2\text{TP} + \text{FP} + \text{FN}}$

where TP, FP, and FN are true positives, false positives, and false negatives at the rendered character level.

What it computes: The F1-score computed over visualized character instances rather than LaTeX tokens — two LaTeX strings that render identically but differ syntactically (e.g., $x^2$ vs. $x^{2}$ ) are compared by matching their rendered character positions. TP counts characters that appear in the same spatial position in both rendered prediction and ground truth.

Why this form: Edit-distance on raw LaTeX strings is "highly sensitive to syntactic variations" and penalizes equivalent representations. CDM is "more structure-aware and representation-invariant" because it evaluates at the semantic (visual) level rather than the syntactic (markup) level. The F1 formulation (harmonic mean of precision and recall) balances missed characters against hallucinated characters.

Table Recognition Metrics (Appendix I.1). The core metric is Tree-Edit-Distance-based Similarity (TEDS):

$\text{TEDS}(T_1, T_2) = 1 - \frac{\text{TED}(T_1, T_2)}{\max(\text{size}(T_1), \text{size}(T_2))}$

where $T_1$ and $T_2$ are the predicted and ground-truth HTML trees, TED is the tree edit distance (minimum number of node insertions, deletions, and substitutions to transform one tree into the other), and size is the number of nodes.

What it computes: The similarity score normalizes the tree edit distance by the size of the larger tree and subtracts from 1, yielding a value in $[0, 1]$ where 1 indicates identical tree structure and content. The tree edit distance jointly considers structural tags (table, tr, td) and cell content.

Why this form: String-level edit distance on the linearized HTML cannot explicitly capture hierarchical structure — a mis-nested table with correct content receives a misleadingly low edit distance. TEDS operates on the tree structure directly, so structural errors (misplaced rows, incorrect spanning) are penalized regardless of content accuracy. S-TEDS "further simplifies TEDS by ignoring cell content and focusing solely on logical structure (row, column, and spanning relations)," making it suitable for structure-centric benchmarking where content recognition is evaluated separately.

Benchmarks (Appendix I.2). The paper summarizes benchmarks across sub-tasks and holistic evaluation in Table S1, organized by the task they evaluate. Key benchmarks include:

Layout analysis: DocBank (400K samples, token-level annotations for 13 categories), D4LA (2,224 noisy document images, 27 categories), DocLayNet (80,863 pages, 11 categories, 7 document types), M6Doc (74 categories, bilingual, multiple formats).
Mathematical expression recognition: CROHME (handwritten, ~3K test samples), UniMER-Test (23,757 samples, 4 sub-tasks, printed and handwritten).
Table recognition: PubTabNet (9K scientific tables, HTML annotations), FinTabNet (10K financial tables).
Holistic document parsing: OmniDocBench (9 document types, 981 samples, multi-level annotations for end-to-end, task-specific, and attribute-based evaluation), OmniDocBench v1.5 (expanded to 1,355 samples), CC-OCR (39 sub-tasks, 7,058 samples, multilingual), OCRBench v2 (31 sub-tasks, 1,500 samples), olmOCR-Bench (1,402 samples, unit-test-based evaluation), Real5-OmniDocBench (1,355 samples with 5 perturbation types including scanning, warping, screen photography, illumination variation, and skew).

The paper notes a trajectory from "isolated, subtask-driven datasets toward unified, multimodal, and robustness-aware evaluation frameworks," reflecting "a growing recognition that accurate document parsing must be assessed not only by surface-level string similarity, but also by structural fidelity, interpretative flexibility, and resilience under real-world conditions."

Datasets Consolidation (Appendix II)

The paper provides detailed tables cataloging datasets for each sub-task:

Layout analysis (Table S2): 16 datasets spanning historical documents (IMPACT, Saint Gall), complex typographic layouts (PubLayNet, DocLayNet, M6Doc), and domain-specific collections (CORD for receipts, FUNSD for forms).
OCR (Table S3): 29 datasets organized by task (TD: text detection, TR: text recognition, TS: text spotting), covering scene text (ICDAR series, Street View Text), irregular text (CUTE80, Total-Text), synthetic data (SynthText with 800K images, SynthAdd with 1.2M), and specialized scenarios (WordArt for artistic text, ICDAR2019-ReCTS for Chinese signboards).
Mathematical expression detection and recognition (Table S4): 12 datasets including UW-III (100 pages, inline and displayed), Marmot (594 pages), ICDAR-2021 IBEM (8,900 images, 166K expressions), Im2Latex-100K (103,556 printed formulas), CROHME (12,178 handwritten), HME100K (99,109 handwritten), and UniMER-1M (1,061,791 printed and handwritten).
Table detection and structure recognition (Table S5): 28 datasets spanning government documents (ICDAR2013), academic papers (TABLE2LATEX-450K, PubTables-1M, SciTSR), financial tables (FinTabNet, FinTab), multilingual tables (WikiTableSet with 50M samples), and challenging real-world scenarios (WTW with 14,582 pages of photos, files, and web pages).

4. Key Insights and Innovations

Innovation 1: A Unified Taxonomic Framework That Resolves the Pipeline-VLM Dichotomy as Complementary Rather Than Sequential

The paper's most fundamental intellectual contribution is not any individual method or finding, but the meta-framework — the taxonomic structure itself — that organizes the entire field of document parsing along an architectural axis (modular pipeline vs. unified VLM) while tracking temporal evolution. This reframing changes how researchers and practitioners think about the relationship between historical pipeline components and modern VLM systems: rather than treating VLMs as a replacement that makes pipelines obsolete, the taxonomy reveals them as two paradigms with complementary design philosophies that are increasingly converging in hybrid architectures.

What was the dominant assumption? Prior to this survey, the literature implicitly treated the timeline as progressive: rule-based methods → deep learning pipelines → end-to-end VLMs, with each phase superseding the previous one. Task-specific surveys studied individual pipeline components (layout analysis, OCR, table recognition) in isolation. System-level surveys like Subramani et al. (2020) and Arlazarov et al. (2022) described the pipeline era but were written before VLMs reshaped the field, so they offered no framework for understanding the transition. Early VLM papers (Nougat, GOT-OCR2.0) presented their approaches as paradigm-shifting breakthroughs that rendered modular pipelines unnecessary, without analyzing what was gained or lost relative to the pipeline approach's decades of engineering. The field had a missing conceptual bridge: no shared vocabulary or organizing principle for comparing a Faster R-CNN-based table detector from 2017 with a 2B-parameter VLM trained end-to-end in 2025.

What makes this taxonomy distinctive as an idea? The survey's architectural axis is not merely a classification system — it is an analytical lens that reveals non-obvious relationships across the literature. By organizing methods according to how they decompose the parsing problem rather than what task they solve, the taxonomy makes visible three patterns that were previously obscured:

First, it reveals the intellectual lineage between pipeline components and modern VLM sub-architectures. The multi-stage specialized VLMs (MonkeyOCR, MinerU 2.5, GLM-OCR) are not a retreat from the VLM paradigm — they are a deliberate re-introduction of pipeline-style decomposition within unified neural architectures, motivated by the efficiency and hallucination limitations of purely end-to-end generation. The taxonomy captures this by placing multi-stage models as a distinct category within VLMs rather than treating them as "not fully end-to-end." This is a genuinely non-obvious organizational choice: it argues that structural decomposition is a design dimension orthogonal to the use of shared vision-language representations, and that the field's future likely lies in exploring this trade-off space rather than converging on either extreme.

Second, it provides a common evaluation framework that enables cross-paradigm comparison. Table 2 is the survey's most empirically significant contribution because it brings general-purpose VLMs, end-to-end specialized VLMs, and multi-stage specialized VLMs onto the same benchmark (OmniDocBench-v1.5) and reveals that multi-stage models with 0.9B-2B parameters substantially outperform general-purpose models with 72B+ parameters on structure-sensitive metrics (FormulaCDM, TableTEDS, Reading-Order Edit Distance). This finding is not obvious a priori — many practitioners would assume that larger, more general models dominate — and it has direct implications for deployment decisions. The taxonomy provides the conceptual apparatus that makes this comparison meaningful: without the three-way VLM classification, one might simply compare "VLM vs. pipeline" and miss the critical multi-stage vs. end-to-end distinction within VLMs.

Third, it establishes architectural distinguishability as the primary classification principle rather than task, dataset, or supervision strategy. This is a deliberate theoretical choice (Section 3.2, Principle 1) that reflects a conviction about what drives system behavior: "architectural design largely determines how visual features, textual content, and structural representations are integrated." A Faster R-CNN table detector and a DETR-based table detector differ primarily in architecture (CNN-based vs. Transformer-based), not in task or training signal, and their architectural choices determine their failure modes (locality bias vs. quadratic complexity). The taxonomy follows through on this principle consistently across all sub-tasks, creating a framework where the same architectural categories (CNN-based, Transformer-based, encoder-decoder, autoregressive) appear in every component section, enabling cross-task architectural comparison.

Where is the evidence? The taxonomy's intellectual merit is demonstrated by the survey it enables. The comprehensive organization of approximately 230 papers into a coherent narrative — from rule-based layout analysis in the 1990s to RLVR-trained end-to-end VLMs in 2025 — would be impossible without the taxonomic scaffold. The paper's ability to track how specific technique families (e.g., image-to-sequence approaches) migrate across tasks — from OCR to mathematical expression recognition to table structure recognition — is a direct result of organizing by architecture rather than by task. The consolidated Table 2, which the authors claim provides "the first consolidated performance comparison of VLM-based document parsers," depends entirely on the VLM sub-classification that distinguishes general-purpose, end-to-end specialized, and multi-stage specialized models.

Significance and limitations. This is a fundamental intellectual contribution, not an incremental one. It provides the organizational scaffold the field has been missing since the VLM paradigm began accelerating in 2023-2024, enabling practitioners to locate specific methods within a coherent structure, researchers to identify under-explored combinations (e.g., PRM-guided search within VLM architectures, which no existing method implements), and benchmark designers to develop evaluation protocols that distinguish architectural trade-offs. The taxonomy is descriptive rather than prescriptive — it does not argue for one paradigm over another — which makes it durable: as new models emerge, they can be placed within the existing framework rather than requiring the framework itself to be revised. A limitation is that the taxonomy's primary axis (pipeline vs. VLM) may blur further as hybrid systems proliferate. The paper acknowledges this implicitly by distinguishing multi-stage from end-to-end VLMs, but the boundary between "pipeline with neural inter-stage communication" and "multi-stage VLM" is inherently fuzzy and may become less useful as a primary organizing principle.

Innovation 2: The Multi-Stage vs. End-to-End Distinction Within VLMs as a Novel Design Dimension

Within the unified VLM paradigm, the paper introduces a sub-classification that represents a genuinely novel conceptual distinction: end-to-end specialized VLMs (which collapse layout detection, content recognition, and structural inference into a single autoregressive sequence) versus multi-stage specialized VLMs (which decompose the process into interpretable sub-tasks within a shared neural architecture). This distinction — and the paper's argument that it represents competing design philosophies rather than a maturity hierarchy — is both intellectually original and practically consequential.

What was the prior framing? Before this survey, the VLM-based document parsing literature implicitly treated end-to-end generation as the ultimate goal and multi-stage approaches as transitional compromises. The narrative embedded in papers like GOT-OCR2.0 ("General OCR Theory: Towards OCR-2.0 via a Unified End-to-End Model") and DeepSeek-OCR (which frames visual tokens as "an efficient compression medium" for direct text generation) was that collapsing the entire pipeline into a single forward pass was inherently superior — simpler architectures, more elegant, no error cascading. Multi-stage systems like MonkeyOCR or MinerU were presented by their authors as engineering solutions, not as principled alternatives, and the language used to describe them often emphasized their pragmatic advantages (efficiency, deployment feasibility) rather than their architectural philosophy.

What makes this distinction intellectually distinctive? The paper reframes multi-stage decomposition not as a temporary compromise on the path to fully end-to-end systems, but as a legitimate and potentially superior design paradigm with its own theoretical motivations. This reframing has three components:

First, the paper identifies a fundamental structural tension in end-to-end generation: "requiring a single autoregressive sequence to encode geometry, textual content, and relational dependencies places considerable demands on model capacity and optimization stability" (Section 9.2). The architecture must simultaneously represent where elements are (layout geometry), what they contain (content recognition), and how they relate (reading order, table structure, cross-references) — all within a single sequence generation objective. This is not merely an engineering challenge; it is a representational one. The one-dimensional autoregressive token stream is a poor match for two-dimensional document structure, and "as document layouts grow more complex and dense, scalability and interpretability remain open challenges."

Second, the paper characterizes multi-stage systems as having a positive design philosophy, not just a deficiency relative to end-to-end models: they "reintroduce controlled decomposition while retaining neural integration" in order to "improve efficiency and reduce hallucination in high-resolution, text-dense scenarios" (Section 9.3). The phrase "controlled decomposition" is important — it implies deliberateness, not failure to achieve end-to-end. The specific mechanisms — the SRR paradigm in MonkeyOCR, the coarse-to-fine inference in MinerU 2.5, the two-stage layout-then-recognition pipeline in GLM-OCR — are described as choices that enable capabilities (block-level parallel recognition, long-document processing, compact parameter counts) that end-to-end models struggle with.

Third, the paper frames this as a trade-off space to be explored, not a rivalry to be resolved. The Discussion (Section 10.1) explicitly states: "The ongoing competition between unified end-to-end modeling and structured hybrid decomposition reflects a broader trade-off between architectural elegance and controlled structural precision." This is a mature intellectual stance — it treats both paradigms as having valid motivations and recognizes that the optimal choice depends on deployment context (computational budget, document type, accuracy requirements) rather than being universally determined.

Where is the evidence? Table 2 provides quantitative support for the multi-stage advantage on specific metrics: GLM-OCR (multi-stage, 0.9B) achieves 94.62 Overall, 93.96 TableTEDS, and 93.90 FormulaCDM — significantly higher than FireRed-OCR (end-to-end, 2B) at 92.94 Overall, 90.31 TableTEDS, and 91.71 FormulaCDM, and far higher than Gemini-3.0 Pro (general-purpose) at 90.33 Overall, 88.28 TableTEDS, 89.18 FormulaCDM. The pattern holds across multiple multi-stage systems (PaddleOCR-VL-1.5 at 94.50 Overall with 0.9B) and multiple end-to-end systems. The efficiency dimension is equally important: multi-stage models achieve these results with 0.9-1.2B parameters compared to 72B+ for general-purpose models, a reduction of ~60-80× in parameter count.

Beyond Table 2, the paper's qualitative analysis of failure modes supports the multi-stage philosophy. The observation that end-to-end models exhibit "reading-order inconsistencies in complex layouts, such as multi-column academic papers" (Section 9.2) is a direct consequence of collapsing spatial structure into a linear sequence — a problem that multi-stage systems explicitly address through separate layout analysis and region-level recognition. The paper's documentation of hallucination in "dense, professionally typeset documents" (Section 9.1) aligns with the multi-stage strategy of constraining recognition to pre-detected regions, preventing the model from generating content for regions that don't exist.

Comparison to prior work. No prior survey or taxonomy has made this distinction. The closest analogue in the broader AI literature is the distinction between end-to-end and modular approaches in robotics, autonomous driving, and other embodied domains — but in document parsing, this architectural debate was happening implicitly in model releases without a structured analysis. The survey provides the analysis for the first time.

Significance. This is a fundamental conceptual contribution that changes how the field should think about VLM design for document parsing. It moves the conversation from "end-to-end is the destination" to "end-to-end and multi-stage are design points in a space defined by the trade-off between global consistency and controlled structural precision." It has direct practical implications: a team building a document parsing system now has a structured framework for choosing between the two paradigms based on their specific constraints (latency, parameter budget, document complexity), and a researcher proposing a new model can position it within this trade-off space rather than simply claiming superior benchmark scores.

A limitation is that the survey does not provide a theoretical framework for predicting when multi-stage decomposition will outperform end-to-end generation. The qualitative arguments about representational mismatch between 2D layout and 1D sequences are plausible but not formalized. Future work might develop an information-theoretic or geometric analysis that characterizes the conditions under which task decomposition in the sequence space is beneficial.

Innovation 3: Causal Visual Flow as a Unifying Principle for Vision-Language Alignment in Document Understanding

Among the architectural innovations surveyed, the paper highlights DeepSeek-OCR 2's "causal visual flow" (Section 9.2) as a conceptually significant departure from standard vision-language alignment in document models. This is not the paper's own innovation — it surveys existing work — but the survey's identification and contextualization of this idea as a cross-cutting principle is intellectually distinctive.

What was the default approach? The standard approach in most VLMs — including both general-purpose models (Qwen2.5-VL, InternVL3) and specialized document models (GOT-OCR2.0, early DeepSeek-OCR) — uses CLIP-style vision encoders that process document images in raster order (left-to-right, top-to-bottom) and produce visual tokens that are fed into a language model. This approach implicitly encodes a fixed spatial prior: visual features are extracted based on proximity in pixel space, and the language model must learn to recover semantic structure (reading order, element relationships) from features that were computed without knowledge of that structure.

What changes with causal visual flow? DeepSeek-OCR 2 introduces DeepEncoder V2, which "replaces CLIP-style encoders with a compact LLM-based architecture that models a 'causal visual flow' — reordering visual features according to semantic dependencies rather than fixed raster order" (Section 9.2). The key insight is that a document's visual structure is not uniformly distributed in pixel space — a figure in the bottom-right corner may be referenced by text in the top-left; an equation's semantic interpretation depends on surrounding text, not just adjacent pixels; a table's header row is semantically prior to its data rows regardless of spatial position. By reordering visual features according to these semantic dependencies — essentially treating document structure as a causal graph rather than a spatial grid — the model directly addresses the mismatch between two-dimensional layouts and one-dimensional token sequences.

Why is this intellectually distinctive? The concept represents a shift from spatial encoding (where position in the image determines processing order) to semantic encoding (where logical relationships determine processing order). This is not just an efficiency trick — it changes the fundamental assumption about how visual information should be represented for language model consumption. Standard approaches force the language model to learn that a patch in position (i, j) might be semantically related to a patch in position (i+100, j-50); causal visual flow encodes that relationship explicitly in the token ordering.

The survey positions this within a broader trajectory: "DeepSeek-OCR reframed the vision-language interface from an LLM-centric perspective, treating visual tokens as an efficient compression medium for textual information" (Section 9.2). This framing — visual tokens as compressed text rather than as image features — is a conceptual inversion. Instead of asking "how do we extract text-relevant features from an image," the model asks "how do we compress the document's textual and structural content into a token sequence that a language model can generate from." The causal visual flow is the mechanism that enables this compression while preserving structural relationships.

Comparison to prior work. Prior document VLMs used vision encoders designed for natural images and adapted them to documents. CLIP was trained on internet photos with captions; ViT was designed for object-centric tasks. The causal visual flow approach is document-native: it is designed specifically for the structural properties of documents (reading order, element hierarchy, cross-references) rather than inheriting architectural biases from general computer vision. This represents a maturation of the field — document parsing is developing its own architectural primitives rather than borrowing from neighboring domains.

Where is the evidence? DeepSeek-OCR 2 achieves 91.09 Overall on OmniDocBench-v1.5 with 3B parameters (Table 2), substantially outperforming much larger general-purpose models. The specific contribution of the causal visual flow to these results cannot be isolated from the survey alone (the ablation is in the DeepSeek-OCR 2 paper, not the survey), but the survey's framing of this architectural choice as a principled response to the fundamental tension between 2D layout and 1D generation establishes it as a conceptual innovation worth tracking independently of its benchmark contribution.

Significance. This is an incremental architectural innovation (not a paradigm shift) but one with significant conceptual implications. It suggests that the field is moving toward document-native architectures that encode structural priors explicitly rather than relying on language models to infer them from image-space features. The survey's role in identifying and contextualizing this trend — connecting it to the broader tension between spatial layout and sequential generation — makes it visible as a research direction rather than an isolated implementation detail.

Innovation 4: Verifier Over-Optimization and Hallucination as the Unifying Bottleneck Across Both Paradigms

The paper identifies a cross-cutting failure mode that limits both modular pipelines and unified VLMs: the tendency for optimization (whether through verifier-guided search in pipelines or autoregressive generation in VLMs) to produce outputs that are structurally plausible but factually incorrect — a phenomenon the paper documents across search-based OCR, formula recognition, table extraction, and VLM generation.

What is the prior framing? Historically, modular pipeline failures were attributed to error cascading — inaccuracies in early stages propagate forward — while VLM failures were attributed to hallucination — generation of plausible but nonexistent content. These were treated as distinct pathologies requiring distinct solutions. Pipeline researchers focused on improving individual module accuracy and robustness; VLM researchers focused on training data quality, decoding strategies, and post-hoc factuality verification.

What the paper reveals. By treating both paradigms within a unified framework, the survey shows that these are manifestations of the same underlying problem: the optimization objective (whether a PRM's correctness score in a pipeline or a next-token prediction loss in a VLM) is a proxy for structural truth, and aggressive optimization against this proxy produces outputs that satisfy the proxy without satisfying the underlying truth. In pipeline systems, this appears as beam search finding solutions that score highly under a PRM but are incorrect (verifier over-optimization). In VLMs, it appears as the model generating plausible content not present in the document (hallucination) and "reading-order inconsistencies in complex layouts" (Section 9.2) because the sequence generation objective rewards local coherence over global structural fidelity.

The paper documents this pattern across multiple sub-tasks:

Formula recognition: Purely sequence-based models "still linearize inherently two-dimensional structures, which limits their ability to represent hierarchical relations explicitly" (Section 6.2.1) — a form of structural hallucination where the generated LaTeX is syntactically valid but structurally incorrect.
Table recognition: Image-to-sequence approaches generate structural markup that can be "sensitive to output linearization choices and may struggle when multiple structural interpretations are plausible" (Section 7.2.3) — the model picks a plausible structure without verifying it against the visual evidence.
Chart parsing: Direct generation makes "numerical faithfulness harder to guarantee, especially for visually cluttered or stylistically unusual charts" (Section 8.1).
End-to-end VLMs: Hallucination is "particularly pronounced in dense, professionally typeset documents such as scientific PDFs, suggesting reliance on statistical priors rather than explicit structural constraints" (Section 9.1).

Why this reframing matters. By identifying a common failure mode across paradigms, the paper provides a unified diagnostic lens. It suggests that improving verifier robustness — whether the verifier is an explicit PRM in a pipeline or the implicit structural constraint enforcement in a VLM's training objective — is the key bottleneck for both paradigms. This is a non-obvious connection: one might expect that the solution to pipeline error cascading is better individual modules, and the solution to VLM hallucination is better training data, but the paper's analysis suggests that both would benefit from explicit structural verification mechanisms. The post-training integration of reinforcement learning (RL) in VLM-based systems — Logics-Parsing's layout-aware rewards, Infinity-Parser's document-level rewards, olmOCR 2.0's RLVR with binary unit tests — can be understood as reintroducing verifier-like structural constraints into the learning process, exactly the mechanism that the pipeline paradigm accesses through explicit PRMs.

Comparison to prior work. Prior surveys either focused on one paradigm's failure modes (pipeline error propagation in early surveys, VLM hallucination in recent model-specific papers) or treated them as unrelated. The survey's unified perspective is novel and productive: it suggests that research on robust verification — which has a long history in pipeline-based document analysis (PRM training, Monte Carlo rollout supervision, best-of-N weighted selection) — may have direct applicability to VLM training and inference that the field has not yet fully explored.

Where is the evidence? The evidence is distributed across multiple sections rather than concentrated in a single figure. The qualitative descriptions of failure modes (repetitive generation in beam search, reading-order inconsistencies in VLMs, hallucination in dense documents) are consistent with the unified bottleneck hypothesis, but the paper does not provide a controlled experiment that isolates verifier quality as the causal factor. This is a conceptual contribution — a reframing of the problem — rather than an empirically validated claim. The survey's value is in identifying this as a cross-cutting issue and suggesting it as a research direction, not in proving it.

Significance. This is a diagnostic reframing more than a technical innovation. Its value is in redirecting research attention: rather than treating pipeline errors and VLM hallucinations as separate problems requiring separate solutions, the paper suggests that progress on verifier robustness — designing training objectives, decoding strategies, and architectural constraints that better align optimization with structural truth — would benefit the entire field. The explicit call in the Discussion (Section 10.1) for "developing specialized reward models for document parsing" as a priority, motivated by the potential for semi-supervised learning on large-scale unlabeled data, flows directly from this diagnosis.

5. Experimental Analysis

Evaluation Methodology

Dataset. The primary holistic evaluation benchmark used to compare VLM-based systems is OmniDocBench-v1.5 [29, 92, 143], which contains 1,355 document pages spanning nine document types with multi-level annotations supporting end-to-end, task-specific, and attribute-based evaluation across layout, reading order, text, formula, and table elements. Additional benchmarks summarized in Table S1 include task-specific datasets for layout analysis (DocBank, DocLayNet, M6Doc), mathematical expression recognition (CROHME, UniMER-Test), table recognition (PubTabNet, FinTabNet), and robustness-oriented datasets (Real5-OmniDocBench with five perturbation types). The paper does not run new experiments; it consolidates reported results from the surveyed papers, primarily on OmniDocBench-v1.5.
Base models. The performance comparison in Table 2 covers three categories: general-purpose VLMs (GPT-4o, GPT-5.2, Gemini-2.5 Pro, Gemini-3.0 Pro, InternVL3-76B/241B, Qwen2.5-VL-72B, Qwen3-VL-2B/235B, Qwen3.5-397B), end-to-end specialized VLMs (Mistral OCR, OCRFlux-3B, POINTS-Reader-3B, olmOCR-7B, MinerU2-VLM-0.9B, Nanonets-OCR-s-3B, dots.ocr-2.7B, DeepSeek-OCR-3B, DeepSeek-OCR 2-3B, OCRVerse-4B, FireRed-OCR-2B), and multi-stage specialized VLMs (Dolphin-0.3M/1.5, MonkeyOCR-pro-1.2B, MonkeyOCR-3B/pro-3B, MinerU2.5-1.2B, PaddleOCR-VL-0.9B, PaddleOCR-VL-1.5-0.9B, GLM-OCR-0.9B). Parameter counts range from 0.3M (Dolphin) to 397B (Qwen3.5). These models span the design spectrum from general-purpose multimodal reasoning to document-specialized architectures, enabling cross-paradigm comparison on structure-sensitive parsing metrics.
Metrics. The evaluation uses a multi-dimensional suite. TextEdit measures normalized edit distance between predicted and ground-truth text. FormulaCDM (Character Detection Matching) evaluates mathematical expression recognition at the rendered-character level as CDM = 2TP/(2TP + FP + FN), capturing structural fidelity independent of LaTeX syntactic variation. TableTEDS (Tree-Edit-Distance-based Similarity) measures HTML tree similarity between predicted and ground-truth table structures as TEDS = 1 − TED(T₁, T₂)/max(size(T₁), size(T₂)), accounting for both structural tags and cell content. TableTEDS_s is the structure-only variant (S-TEDS) that ignores cell content and evaluates only logical structure. Reading-Order Edit Distance (R-orderEdit) measures the normalized edit distance of the reading order sequence. Overall is an aggregated score combining these dimensions. For layout detection and element localization, the paper summarizes standard metrics including IoU, mAP, and the Page-IoU metric proposed in MinerU 2.5 (Appendix I.1).
Baselines. The experimental framework does not define baselines in the traditional sense — the survey is a comparative analysis rather than a controlled experiment. However, several implicit baselines serve as reference points in Table 2 and throughout the paper: (1) general-purpose VLMs (GPT-4o, Qwen2.5-VL-72B, etc.) serve as baselines for specialized document VLMs, demonstrating what broader multimodal models achieve on parsing tasks without task-specific optimization; (2) Majority voting (Section 5.2 of prior sections) serves as a verifier-free baseline for answer selection in search-based methods, where the most common final answer among N sampled solutions is selected without any learned verifier; (3) Parallel sampling (best-of-N) serves as the baseline for sequential revision strategies (Section 6 of prior sections), where N independent solutions are generated and the best selected via verifier or majority; (4) ORM best-of-N weighted serves as a baseline for PRM-based search, comparing outcome-level verification against process-level verification.
Generation budget / compute accounting. In the search and revision results described in prior sections, the unit of compute is generations — one complete sampled answer from the base model. For beam search, the budget equals the number of beams N. For lookahead search with k lookahead steps, the cost is N × (k+1) to account for additional rollout computation. For revisions, the budget is distributed across sequential and parallel dimensions: fully parallel uses N independent chains, fully sequential uses one chain of N revisions, and hybrid uses √N parallel chains each of length √N. Budgets are typically swept across powers of 2 from 2⁰ to 2⁹ (1 to 512 generations). In the FLOPs-matched comparison (Section 7 of prior sections), total compute is measured in FLOPs using the pretraining formula X = 6ND_pretrain and inference formula Y = 2ND_inference, with three ratios of inference-to-pretraining tokens tested: R ≪ 1 (R = 0.16), R ≈ 1 (R = 0.79), and R ≫ 1 (R = 22). For Table 2, there is no uniform compute accounting — models are compared by their parameter count and reported benchmark scores, without controlling for inference FLOPs or training compute.
Cross-validation / statistical protocol. For the compute-optimal strategy selection in prior sections, two-fold cross-validation is used within each difficulty bin on the 500-question MATH test set. The best-performing strategy is selected on one fold and evaluated on the other, with results averaged, to avoid overfitting the policy to the test set. Difficulty bins are based on 2048 samples per question. For Table 2, no cross-validation protocol is described — results are single-point benchmark scores reported from each model's original publication or evaluation, without confidence intervals or significance testing. The paper does not report variance across runs, statistical tests comparing model pairs, or uncertainty estimates for any benchmark numbers.

Main Quantitative Results

5.2.1 VLM Comparison on OmniDocBench-v1.5 (Table 2)

The headline finding from Table 2 is that multi-stage specialized VLMs with compact parameter counts outperform both general-purpose VLMs and end-to-end specialized VLMs on structure-sensitive document parsing metrics, with the smallest models in this category achieving the highest Overall scores.

General-purpose VLMs (Table 2, rows 1-11): The best general-purpose model, Gemini-3.0 Pro (parameter count not publicly reported but likely exceeds 100B), achieves Overall 90.33, with TextEdit of 0.065, FormulaCDM of 89.18, TableTEDS of 88.28, TableTEDS_s of 90.29, and R-orderEdit of 0.071. The best open-source general-purpose model shown is Qwen3.5-397B at 90.80 Overall (with dashes for sub-metrics, implying the full breakdown was not reported). The pattern across general-purpose models shows that scaling parameter count improves performance — Qwen3-VL-2B achieves 81.87 Overall while Qwen3-VL-235B achieves 89.15 — but the returns diminish substantially, and even the largest general models fail to surpass specialized models on structure-sensitive metrics. For instance, Gemini-3.0 Pro's TableTEDS of 88.28 is 5.68 points below GLM-OCR's 93.96, and its FormulaCDM of 89.18 is 4.72 points below GLM-OCR's 93.90, despite Gemini-3.0 Pro presumably using far more parameters.

End-to-end specialized VLMs (Table 2, rows 12-22): Within this category, FireRed-OCR-2B achieves the highest Overall score at 92.94, with the best sub-metrics across the category: TextEdit of 0.032, FormulaCDM of 91.71, TableTEDS of 90.31, TableTEDS_s of 93.81, and R-orderEdit of 0.041. DeepSeek-OCR 2-3B follows closely at Overall 91.09, with TextEdit of 0.048, FormulaCDM of 90.31, TableTEDS of 87.75, and TableTEDS_s of 92.06. The weaker end-to-end models — Mistral OCR at 74.82 Overall, OCRFlux-3B at 74.82, POINTS-Reader-3B at 80.98 — demonstrate substantial variance within this category, suggesting that end-to-end performance depends heavily on training data quality, architecture design, and optimization rather than being inherently strong as a paradigm. The progression from earlier end-to-end models (Mistral OCR at 74.82) to more recent ones (FireRed-OCR at 92.94, chronologically 2025.3 to 2026.3) shows rapid improvement, with approximately 18 points of Overall gain in roughly one year.

Multi-stage specialized VLMs (Table 2, rows 23-30): This category achieves the highest absolute performance. GLM-OCR-0.9B achieves Overall 94.62 — the highest reported value — with TextEdit of 0.030, FormulaCDM of 93.90, TableTEDS of 93.96, TableTEDS_s of 96.39, and R-orderEdit of 0.044. PaddleOCR-VL-1.5-0.9B is a close second at Overall 94.50, with TextEdit of 0.035, FormulaCDM of 94.21, TableTEDS of 92.76, and TableTEDS_s of 95.79. This means two models with only 0.9B parameters each outperform every general-purpose VLM (including models with 200-400× more parameters) and every end-to-end specialized VLM (including models with 2-3× more parameters) on the Overall score. The parameter efficiency is even more striking when considering the gap between the earliest multi-stage model (Dolphin at 74.67 with 0.3M parameters) and the latest (GLM-OCR at 94.62) — nearly 20 points of improvement achieved by refined decomposition strategies within roughly the same compact parameter regime.

Cross-category comparison: Comparing across the three categories at the same or similar overall score levels reveals the structural trade-offs. GPT-5.2 achieves 85.50 Overall versus Dolphin-1.5 at 83.21 — a general-purpose model with likely >100B parameters versus a multi-stage model with 0.3M parameters, separated by only ~2.3 points, with the multi-stage model being roughly 300,000× smaller in parameter count. At the high end, the gap between the best general model (Qwen3.5-397B at 90.80) and the best multi-stage model (GLM-OCR-0.9B at 94.62) is 3.82 points, with a parameter ratio exceeding 400:1 in favor of the smaller model. For end-to-end vs. multi-stage, the best end-to-end (FireRed-OCR-2B at 92.94) trails the best multi-stage (GLM-OCR-0.9B at 94.62) by 1.68 points, representing a meaningful but smaller gap than the general-purpose comparison.

Metric-specific patterns: The largest gaps between multi-stage and other paradigms appear on table-specific metrics. GLM-OCR's TableTEDS of 93.96 exceeds FireRed-OCR-2B's 90.31 by 3.65 points and Gemini-3.0 Pro's 88.28 by 5.68 points. For table structure specifically (TableTEDS_s), GLM-OCR at 96.39 exceeds FireRed-OCR at 93.81 by 2.58 points. Reading order (R-orderEdit) is consistently excellent across top models: GLM-OCR at 0.044, PaddleOCR-VL-1.5 at 0.043, FireRed-OCR at 0.041, and Gemini-3.0 Pro at 0.071 — all relatively low, but the specialized models maintain an edge. Formula recognition (FormulaCDM) shows the strongest multi-stage advantage: GLM-OCR at 93.90 outperforms FireRed-OCR at 91.71 by 2.19 points and Gemini-3.0 Pro at 89.18 by 4.72 points, and PaddleOCR-VL-1.5 achieves 94.21 — the highest individual metric across all models and categories.

5.2.2 Task-Specific Benchmark Results (Sections 4-8, Tables S2-S5)

The paper's component-level surveys (Sections 4-8) report quantitative results distributed across the literature rather than from new experiments. Key patterns include:

Layout analysis (Section 4): The paper reports that Transformer-based methods demonstrate strong performance on several document analysis tasks, while faster YOLO-based models like PP-DocLayout achieve inference speeds exceeding 120 pages per second supporting 23 layout categories, demonstrating real-time feasibility. No consolidated performance table is provided for layout analysis. The M6Doc dataset with 74 fine-grained categories and over 200,000 annotations establishes a comprehensive benchmark for logical layout analysis, but the paper does not report a leaderboard of methods on this dataset.

OCR (Section 5): No consolidated quantitative table is presented. The paper reports that general-purpose models "can often achieve highly competitive OCR performance as a byproduct of multimodal alignment," and that specialized OCR research now focuses on extreme scenarios, interpretability, and reliability rather than benchmark competition.

Mathematical expression recognition (Section 6.2): The UniMER-Test benchmark contains 23,757 samples across 4 sub-tasks covering printed and handwritten scenarios. The MathWriting dataset introduces over 600k handwritten expressions. The paper reports that structure-aware models like TAMER improve hierarchical fidelity but does not provide consolidated performance numbers across methods. PP-FormulaNet reports both higher accuracy and faster inference on a large-scale corpus from arXiv papers. No single leaderboard table aggregates mathematical expression recognition methods.

Table detection and recognition (Section 7): PubTabNet provides 568,000 training samples with HTML annotations for scientific tables; FinTabNet provides 112,887 samples for financial tables. The paper reports that detection has "achieved relatively mature performance on many benchmarks" and that research efforts have shifted toward structure recognition. Unified frameworks like UniTable leverage self-supervised pretraining on unlabeled table images for improved generalization. No consolidated performance table is provided for table recognition methods.

Visual element parsing (Section 8): For chart parsing, ChartAssistant's chart-to-table pretraining and TinyChart's visual token merging are described qualitatively; the Chart2Code benchmark introduces a hierarchical evaluation with three progressively challenging tasks. For OCSR, the paper reports that MLLM-based approaches like MolParser and MolSight achieve robust performance on varied drawing styles and stereoisomer recognition, but no quantitative results are consolidated.

The absence of consolidated quantitative comparisons for individual subtasks represents a significant gap relative to the systematic benchmarking provided for VLM-based holistic parsing (Table 2). Task-specific researchers cannot use this survey to directly compare, for example, the FormulaCDM of TAMER vs. CoMER vs. Uni-MuMER on the same benchmark, because such a comparison is not provided.

Ablation Studies and Robustness Checks

Since this is a survey paper that reports results from other papers rather than conducting new experiments, the "ablation" analysis consists of cross-paradigm comparisons and analysis of metric-specific performance patterns that reveal which design choices drive performance differences.

General-purpose vs. specialized VLMs (Table 2): Comparing models with similar parameter counts across categories reveals the impact of domain specialization. Qwen3-VL-2B (general-purpose) achieves 81.87 Overall while FireRed-OCR-2B (end-to-end specialized) achieves 92.94 — a gain of 11.07 points from the same parameter budget through task-specific architecture and training. This is not a controlled ablation (the models differ in architecture, training data, and optimization procedure) but it demonstrates that general visual-linguistic alignment is insufficient for fine-grained document parsing.
End-to-end vs. multi-stage within specialized VLMs (Table 2): The clearest architectural ablation is between end-to-end and multi-stage models at comparable parameter scales. At ~3B parameters: POINTS-Reader-3B (end-to-end) achieves 80.98 Overall while MonkeyOCR-3B (multi-stage) achieves 87.13 — a gain of 6.15 points. At ~1B parameters: MinerU2-VLM-0.9B (end-to-end) achieves 85.56 Overall while PaddleOCR-VL-0.9B (multi-stage) achieves 92.86 — a gain of 7.30 points. At ~3B: DeepSeek-OCR-3B (end-to-end) at 87.36 vs. MonkeyOCR-pro-3B (multi-stage) at 88.85 — a gain of 1.49 points. The multi-stage advantage is consistent across scales but narrows at higher performance levels, suggesting diminishing returns to decomposition as overall quality improves.
Table structure as the differentiating metric (Table 2): TableTEDS shows the largest and most consistent gap favoring multi-stage models. Comparing models at similar Overall scores: Qwen3-VL-235B (general, 89.15 Overall) achieves 86.21 TableTEDS; DeepSeek-OCR 2-3B (end-to-end, 91.09 Overall) achieves 87.75; GLM-OCR-0.9B (multi-stage, 94.62 Overall) achieves 93.96. The gap in TableTEDS between paradigms (7.75 points between best general and best multi-stage) is larger than the gap in Overall score (5.47 points), confirming that table structure recognition is the sub-task where architectural decomposition provides the greatest advantage.
Formula recognition as a multi-stage strength (Table 2): PaddleOCR-VL-1.5-0.9B achieves FormulaCDM of 94.21, exceeding all end-to-end models (best: FireRed-OCR at 91.71) and all general-purpose models (best: Gemini-3.0 Pro at 89.18). The FormulaCDM metric is specifically designed to be invariant to LaTeX syntactic variation (Section I.1), meaning this advantage reflects genuine structural fidelity rather than formatting preference.
Negative result — hallucination in end-to-end models (Section 9.1, 9.2): The paper documents that end-to-end models exhibit "reading-order inconsistencies in complex layouts, such as multi-column academic papers" — a structural hallucination where the generated sequence is locally coherent but globally incorrect. This is not quantified in Table 2 (R-orderEdit values are reported but not broken out by layout complexity), but it represents a systematic failure mode of the sequential generation paradigm that the multi-stage approach explicitly addresses through explicit layout analysis prior to content recognition.
Negative result — ReST^EM degradation (Appendix K of prior sections): In the revision model context, the paper reports that further optimization using ReST^EM caused sequential revision performance to drop to "approximately 33.5% compared to roughly 38.5% at the optimal ratio" at 256 generations. This negative result highlights the sensitivity of revision training to data generation procedures and the risk that on-policy data collection amplifies spurious correlations rather than improving the revision task.

Critical Assessment

Does Table 2 genuinely support the claim that multi-stage VLMs outperform both general-purpose and end-to-end specialized VLMs?

Yes, but with important qualifications that the paper does not fully address. The evidence in Table 2 is clear: GLM-OCR-0.9B at 94.62 Overall exceeds Gemini-3.0 Pro at 90.33 and FireRed-OCR-2B at 92.94. However, several factors complicate the interpretation:

First, training data is not controlled. GLM-OCR, PaddleOCR-VL-1.5, and FireRed-OCR are trained on different datasets with different curation strategies, data volumes, and annotation quality. GLM-OCR's training includes 200M application-aligned pretraining samples; FireRed-OCR uses geometry-aware data generation and a three-stage curriculum; PaddleOCR-VL-1.5 uses the Real5-OmniDocBench for robustness training. It is impossible to determine whether the multi-stage advantage in Table 2 is due to architectural decomposition or superior training data. A fair comparison would control for training data (same documents, same annotations, same data volume) and vary only the architectural paradigm — but no such experiment exists in the surveyed literature.

Second, parameter count is only one dimension of computational cost. GLM-OCR achieves 94.62 Overall with 0.9B parameters, but inference cost is not reported — does the two-stage pipeline (layout detection then parallel region recognition) require more inference FLOPs than a single forward pass of a 2B end-to-end model? Multi-stage models may trade parameter efficiency for increased inference computation through multiple forward passes, and this trade-off is not quantified anywhere in the survey.

Third, the temporal confound is substantial. The best multi-stage models (GLM-OCR, PaddleOCR-VL-1.5) were released in 2026.3 and 2026.2 respectively — later than the best end-to-end model (FireRed-OCR, 2026.3, roughly contemporaneous) but significantly later than the best general-purpose models (Gemini-3.0 Pro, 2025.11; Qwen3.5, 2026.3). The rapid pace of improvement across all categories (~18 points of Overall gain in ~1 year for end-to-end models, ~20 points for multi-stage in the same period) means that temporal recency may explain some of the multi-stage advantage. A snapshot comparison at a single time point does not establish a durable architectural advantage.

Fourth, the OmniDocBench-v1.5 test set is fixed and relatively small (1,355 samples). As the paper itself notes (Section 10.2), "models may inadvertently overfit to these static sets." The multi-stage models that top the leaderboard may be specifically optimized for this benchmark's annotation conventions, document types, or metric weighting. Without evaluation on multiple independent benchmarks, the ranking in Table 2 could reflect benchmark-specific tuning rather than general parsing capability.

Is the paper's claim that modular pipelines remain essential for robustness well-supported?

Partially. The paper argues (Section 10.1, Section 11) that pipeline systems "remain essential for achieving robustness and fine-grained accuracy in complex real-world scenarios." This claim is primarily supported by two pieces of evidence: (1) multi-stage VLMs — which reintroduce pipeline-like decomposition — outperform end-to-end VLMs on structure-sensitive metrics (Table 2), implying that decomposition itself is valuable; and (2) the Real5-OmniDocBench benchmark (Appendix I.2) reveals that "substantial degradation of both specialized systems and MLLMs" occurs under real-world distortions like scanning, warping, and illumination variation.

The second point is underdeveloped: the paper mentions Real5-OmniDocBench as a benchmark that highlights "performance gaps under practical deployment scenarios" (Appendix I.2) but does not report quantitative results from this benchmark. We do not know whether multi-stage models degrade less than end-to-end models under distortions, whether pipeline systems are more robust than either VLM paradigm, or whether certain perturbation types (e.g., skew vs. illumination) differentially affect architectural choices. The claim that pipelines are "essential for robustness" is therefore an assertion grounded in qualitative reasoning about failure modes (error cascading vs. hallucination) rather than empirical comparison.

Does the survey provide sufficient evidence for its claim about verifier over-optimization as a unified bottleneck?

The paper identifies "verifier over-optimization" and "hallucination" as cross-cutting failure modes (Innovation 4 of prior sections), suggesting that both pipeline and VLM paradigms share a common bottleneck: optimization against proxy signals produces structurally plausible but factually incorrect outputs. This is a conceptual claim, not an empirical one. The survey provides qualitative examples — beam search producing degenerate outputs that score highly under PRMs, VLMs hallucinating content in dense scientific PDFs — but no controlled experiment quantifies the relationship between optimization intensity and structural fidelity, compares the over-optimization threshold across paradigms, or demonstrates that improving verifier robustness causally reduces failures in both pipelines and VLMs.

This is the appropriate scope for a survey: identifying a pattern across the literature is valuable even without experimentally validating it. But readers should not mistake the identification of a pattern for proof that it is the primary bottleneck. The survey's value here is in directing future research toward verifier robustness as a hypothesis to test, not in establishing it as fact.

What experiments would strengthen the survey's claims?

Several experiments, had they been included in the surveyed literature or conducted by the survey authors, would substantially strengthen the paper's comparative analysis:

A controlled cross-paradigm comparison: Train VLM-based parsers from all three categories (general-purpose fine-tuned, end-to-end specialized, multi-stage specialized) on identical training data with identical annotations, then evaluate on multiple independent benchmarks. This would isolate architectural paradigm from data quality as the causal factor in performance differences.
Inference cost analysis: For each model in Table 2, report inference FLOPs per page and wall-clock latency on standardized hardware. The current comparison by parameter count alone obscures the fact that multi-stage models may execute multiple forward passes (layout analysis + N parallel region recognitions) while end-to-end models execute one.
Robustness benchmarking across paradigms: Evaluate the top models from each category on Real5-OmniDocBench or similar perturbation benchmarks, reporting degradation per perturbation type (scanning, warping, skew, illumination) and per element category (text, table, formula). This would directly test the claim that certain architectures are more robust to real-world distortions.
Difficulty-stratified evaluation: Report parsing accuracy stratified by document complexity (simple single-column, multi-column with floating elements, densely structured scientific papers) for each category. The paper's qualitative analysis suggests that end-to-end models struggle with complex layouts, but no quantitative stratification supports this.
Statistical significance testing on benchmark results: Since most models are evaluated on the same fixed test set (OmniDocBench-v1.5, 1,355 samples), bootstrap confidence intervals or paired significance tests between model pairs would indicate whether the reported differences (e.g., 94.62 vs. 92.94) are statistically meaningful or within noise.

What genuine weaknesses exist in the experimental evidence?

Single benchmark for consolidated comparison (Table 2): OmniDocBench-v1.5 is the only benchmark where all three categories are compared. While the paper describes many other benchmarks (Table S1), no cross-paradigm comparison is provided for CC-OCR, OCRBench v2, olmOCR-Bench, or task-specific benchmarks. The relative ranking of the three paradigms may not generalize beyond OmniDocBench-v1.5.
No confidence intervals or uncertainty estimates: Table 2 reports point estimates without standard deviations, confidence intervals, or any indication of measurement uncertainty. On a test set of 1,355 samples, differences of 1-2 points in Overall score may fall within sampling error, particularly when models are close (GLM-OCR at 94.62 vs. PaddleOCR-VL-1.5 at 94.50 — a difference of 0.12 points).
Missing models: Table 2 includes a curated set of models but is not comprehensive. Notable omissions might include certain proprietary systems (Azure Document Intelligence, Amazon Textract, Google Document AI as standalone parsers rather than the Gemini models included) and open-source systems that did not report OmniDocBench-v1.5 results at the time of the survey. The selection criteria for which models appear in Table 2 are not explicitly stated.
Task-specific sections lack quantitative consolidation (Sections 4-8): The survey provides detailed architectural descriptions for layout analysis, OCR, formula recognition, table recognition, and visual element parsing, but does not provide leaderboard tables, performance comparisons, or quantitative trends for any of these sub-tasks. A reader interested in whether TAMER outperforms BTTR on CROHME, or whether YOLO-DLA outperforms DocLayout-YOLO on DocLayNet, must consult the original papers. The survey's value for task-specific practitioners is therefore primarily architectural and conceptual rather than quantitative.
No longitudinal analysis of benchmark saturation: The paper notes that table detection has "achieved relatively mature performance on many benchmarks" (Section 7.1) but does not quantify this — what are the current SOTA mAP scores, how close are they to ceiling, and which specific failure cases remain? Without this analysis, the claim of "mature performance" is vague and potentially misleading.

6. Limitations and Trade-offs

Limitation 1: The Taxonomy's Primary Axis Is Descriptive, Not Predictive — It Cannot Tell You Which Paradigm to Choose for a Given Deployment

The assumption or constraint. The survey organizes the field along an architectural axis — modular pipeline-based systems versus unified VLMs, with the latter further split into end-to-end and multi-stage specialized models — and treats this as the fundamental organizing principle. The taxonomic categories are defined by how systems decompose the parsing problem (explicit task boundaries vs. shared representations vs. controlled decomposition within unified architectures). The paper explicitly positions this as a descriptive framework: it "distinguishes between two dominant paradigms" and organizes methods "according to their architectural structure" (Section 3.3). It does not claim to provide a decision procedure for selecting one paradigm over another.

The consequence. A practitioner reading this survey to decide whether to deploy a modular pipeline, an end-to-end VLM, or a multi-stage VLM for their specific document parsing task receives rich architectural descriptions but no principled guidance for making the choice. The survey identifies the trade-offs qualitatively — pipelines offer "modularity, interpretability, and controllable optimization" with the cost of "error cascading" and inability to perform "end-to-end global optimization" (Section 10.1); end-to-end VLMs offer "global consistency and simplified system design" at the cost of "reading-order inconsistencies" and "hallucination in dense, professionally typeset documents" (Section 9.1, 9.2); multi-stage VLMs offer "improved efficiency, controllability, and long-page robustness" at the cost of increased "system complexity" and potential "inter-stage dependency issues" (Section 9.3) — but these trade-offs are never quantified. There is no characterization of when each paradigm fails, no threshold of document complexity or noise level that predicts cross-over in relative performance, and no cost model that maps deployment constraints (latency budget, parameter budget, document type distribution) to paradigm selection. The taxonomy tells you what exists but not what to use.

What evidence exists in the paper. Table 2 provides the only quantitative cross-paradigm comparison, showing that multi-stage models (GLM-OCR, PaddleOCR-VL-1.5) achieve the highest Overall scores on OmniDocBench-v1.5, followed by end-to-end specialized models (FireRed-OCR, DeepSeek-OCR 2), followed by general-purpose VLMs (Gemini-3.0 Pro, Qwen3.5). However, this is a single-point comparison on a single benchmark, and the paper does not stratify the results by document type, layout complexity, language, or noise condition within OmniDocBench-v1.5. We do not know whether the multi-stage advantage holds for simple single-column documents (where end-to-end models might be equally accurate and faster), for noisy scanned documents (where pipeline robustness might dominate), or for documents in languages with complex typography. The survey acknowledges this implicitly in Section 10.2: "evaluation schemes that reflect quality in actual industrial application scenarios remains a shared priority" — but it does not provide such evaluation itself.

Mitigation status. The paper does not attempt to resolve this limitation. It acknowledges the trade-off as an open question: "The ongoing competition between unified end-to-end modeling and structured hybrid decomposition reflects a broader trade-off between architectural elegance and controlled structural precision" (Section 10.1). Future work on task-conditioned or difficulty-conditioned model selection — analogous to the compute-optimal test-time scaling strategies described in other domains — could address this gap, but the survey does not develop such a framework.

Limitation 2: No Unified Compute or Latency Accounting — Parameter Count Alone Is an Incomplete Metric That Obscures Deployment Feasibility

The assumption or constraint. The VLM comparison in Table 2 reports only parameter count as a measure of model size and computational cost. Inference FLOPs, wall-clock latency, memory footprint, and throughput (pages per second) are not reported for any model. The training data volume and pretraining compute are similarly absent. The paper implicitly treats parameter count as a sufficient proxy for efficiency — the claim that multi-stage models with 0.9B parameters outperform general-purpose models with 72B+ parameters implies an efficiency advantage — but parameter count alone does not determine deployment cost.

The consequence. A multi-stage model like GLM-OCR with 0.9B parameters may execute multiple forward passes: PP-DocLayout-V3 for layout analysis (which itself is a separate model invocation, though the survey does not specify its parameter count), followed by parallel region-level recognition for each detected element (text blocks, tables, formulas, figures). If a page contains 20 elements, the system potentially runs 21 forward passes (1 layout + 20 region recognitions). An end-to-end model like FireRed-OCR with 2B parameters runs a single forward pass. The inference FLOPs comparison is therefore: 0.9B × (N_layout_passes + N_region_passes) versus 2B × 1. For a sufficiently large N_region_passes, the "smaller" multi-stage model may consume more total FLOPs than the "larger" end-to-end model. The same logic applies to latency: parallel region recognition can hide some cost if sufficient hardware parallelism is available, but the layout analysis stage is inherently serial and adds fixed latency to every page. Without reporting these operational characteristics, the efficiency claim based on parameter count is incomplete and potentially misleading.

For general-purpose VLMs, the situation is even more skewed. A model like Qwen3-VL-2B (81.87 Overall) has only 2B parameters but achieves much lower accuracy than GLM-OCR-0.9B (94.62). The "efficiency" lesson appears to be that specialized architectures dominate regardless of parameter count — but the training cost to produce GLM-OCR (curating 200M application-aligned samples, designing the two-stage pipeline, training layout and recognition components) is not amortized in the comparison. A deployment team choosing between "fine-tune a 72B general model" and "build a specialized 0.9B multi-stage system from scratch" faces very different engineering costs that the survey does not acknowledge.

What evidence exists in the paper. The only cost-related numbers in the paper are parameter counts in Table 2 and the latency figure for PP-DocLayout ("real-time inference speeds exceeding 120 pages per second," Section 4.1.1) — which applies to the layout detection component only, not to end-to-end parsing. The paper does mention computational overhead as a limitation of general-purpose VLMs: "the computational overhead associated with multi-billion-parameter models and quadratic attention over high-resolution visual tokens limits their scalability in industrial pipelines" (Section 9.1). But it provides no numbers — no FLOPs estimates, no latency benchmarks, no throughput measurements — for any model category. The multi-stage models' own inference costs are never quantified.

Mitigation status. The paper does not address this limitation. Section 10.1 acknowledges that "the deployment and inference costs associated with VLMs cannot be ignored" and that "most specialized models therefore focus on maximizing precision while reducing parameter counts and deployment overhead," but no metric is proposed for evaluating deployment cost beyond parameter count. The lack of standardized cost reporting is a field-wide issue — most papers report only parameter count and accuracy — but a survey positioning itself as a practical reference for practitioners should flag this gap more prominently and propose standardized reporting conventions.

Limitation 3: OmniDocBench-v1.5 Is the Only Cross-Paradigm Benchmark — Generalization to Other Document Types, Domains, and Conditions Is Unverified

The assumption or constraint. The only quantitative cross-paradigm comparison in the survey is Table 2, which reports performance on OmniDocBench-v1.5 — a benchmark of 1,355 document pages spanning nine document types with "printed/scanned/handwritten" acquisition formats (Table S1). The survey validates its central taxonomic claim — that multi-stage specialized VLMs outperform both general-purpose and end-to-end specialized VLMs — on this single benchmark. The paper does not report cross-paradigm comparisons on any other benchmark in its extensive catalog: not on CC-OCR (7,058 samples, 39 sub-tasks), not on OCRBench v2 (1,500 samples, 31 sub-tasks), not on olmOCR-Bench (1,402 samples, unit-test-based evaluation), and not on task-specific benchmarks like PubTabNet, CROHME, or DocLayNet.

The consequence. The ranking observed on OmniDocBench-v1.5 may not generalize. Different benchmarks have different document type distributions, annotation conventions, and metric weightings. OmniDocBench-v1.5 emphasizes academic and formal documents with relatively clean layouts; it may underrepresent heavily distorted scans, mobile photographs, hand-drawn elements, or domain-specific formats (patents, medical records, engineering diagrams). If multi-stage models are specifically designed to handle complex layouts (through explicit layout analysis) and formal document structures (through region-level recognition), they may excel on OmniDocBench precisely because its distribution aligns with their design assumptions, while performing worse on benchmarks that emphasize different challenges. Conversely, real-world benchmarks like Real5-OmniDocBench (which introduces scanning, warping, screen photography, illumination variation, and skew) are mentioned in Appendix I.2 but no results are reported — we do not know whether the multi-stage advantage persists under physical distortions.

The benchmark size is also a concern: 1,355 test samples, when evaluating differences of 1-2 points in Overall score between top models (e.g., GLM-OCR at 94.62 vs. PaddleOCR-VL-1.5 at 94.50), may not provide sufficient statistical power to distinguish model quality from sampling noise. The paper reports no confidence intervals, no bootstrap estimates, and no significance tests.

What evidence exists in the paper. The paper itself acknowledges the benchmark coverage limitation in Section 10.2: "Although benchmarks like OmniDocBench and OLM-Bench have emerged, they still struggle to cover the infinite variety of documents, and models may inadvertently overfit to these static sets." This is a direct acknowledgment that Table 2's rankings may reflect benchmark-specific optimization. The survey also notes that "developing evaluation schemes that reflect quality in actual industrial application scenarios remains a shared priority" — implying that current benchmarks, including OmniDocBench-v1.5, do not fully capture real-world performance. The lack of cross-benchmark comparison is a gap in the survey's empirical coverage: the very consolidation of evaluation protocols that the paper advocates (Section 1) is not applied to its own central empirical claim.

Mitigation status. The paper partially mitigates this by cataloging a wide range of benchmarks in Table S1 and Appendix I.2, providing the infrastructure for future cross-benchmark evaluation. It also advocates for "more comprehensive evaluation frameworks that better reflect real-world document complexity and multimodal reasoning requirements" (Section 11). But it does not itself conduct such evaluation. The survey's primary empirical contribution — Table 2 — remains single-benchmark evidence for a claim (multi-stage > end-to-end > general-purpose) that the paper presents as a general finding about architectural paradigms.

Limitation 4: Task-Specific Sections Lack Consolidated Quantitative Comparisons — The Survey Is an Architectural Map, Not a Performance Reference for Sub-Task Practitioners

The assumption or constraint. The survey's deep dives into modular pipeline components — layout analysis (Section 4), OCR (Section 5), mathematical expression recognition (Section 6), table detection and recognition (Section 7), and visual element parsing (Section 8) — provide detailed architectural descriptions and track methodological evolution, but they do not include consolidated performance tables or quantitative comparisons between methods within each sub-task. The paper describes how methods work, not how well they work relative to each other. A reader interested in the current state-of-the-art for mathematical expression recognition on CROHME, or the relative mAP of YOLO-DLA versus DocLayout-YOLO on DocLayNet, will not find that information in this survey.

The consequence. The survey serves different audiences unequally. For a researcher or practitioner trying to understand the landscape of document parsing — what approaches exist, how they relate, what the major architectural trends are — the survey is comprehensive and valuable. For a practitioner trying to select a component for a specific sub-task — "which table detector should I use for financial documents?" — the survey provides qualitative guidance (architectural families, representative methods) but no quantitative basis for comparison. The lack of performance tables is particularly consequential for sub-tasks where methods are evaluated on different benchmarks with different metrics (e.g., PubTabNet TEDS vs. FinTabNet TEDS, CROHME ExpRate vs. UniMER-Test CDM), making cross-method comparison challenging even for experts. A consolidated leaderboard — or even a structured summary of which methods achieve what scores on which benchmarks — would substantially increase the survey's practical utility for component selection within pipeline systems.

This gap also means the survey cannot quantify progress within sub-tasks. The paper states that table detection has "achieved relatively mature performance on many benchmarks" (Section 7.1), but provides no numbers: what is the current SOTA mAP, what is the ceiling (human performance or annotation consistency), and what specific failure cases remain? Similarly, it reports that structure-aware formula recognition models "improve hierarchical fidelity" (Section 6.2.2) but does not quantify the improvement — is it a 2% absolute gain on CROHME or a 15% gain? Without quantitative trends, the reader cannot assess whether a sub-task is stalled (needing fundamental breakthroughs) or steadily improving (where engineering refinement suffices).

What evidence exists in the paper. The contrast between the VLM section (Section 9) and the task-specific sections (Sections 4-8) is stark. Section 9 provides Table 2 with 30 models evaluated on 6 metrics. Sections 4-8 provide architectural taxonomies and methodological descriptions with no quantitative tables whatsoever. The paper does summarize benchmark datasets for each sub-task (Tables S1-S5 in the Appendix) and discusses metrics (Appendix I.1), but never maps methods onto these benchmarks. The survey is consistent in its descriptive focus — the taxonomy principles (Section 3.2) prioritize "architectural distinguishability" and "historical fidelity" over quantitative comparison — but this consistency comes at the cost of practical utility for sub-task practitioners.

Mitigation status. The paper does not attempt to add quantitative comparisons to the task-specific sections. The scope of consolidating evaluation protocols (Section 10.2's call for "specialized evaluation frameworks") is acknowledged as future work. A reader who needs sub-task performance comparisons must consult the original papers or task-specific surveys (which the paper cites: Binmakhashen and Mahmoud 2019 for layout analysis, Kasem et al. 2024 for tables, Aggarwal et al. 2022 for formulas). The survey is transparent about this: it positions itself as a holistic survey addressing the gap between fragmented task-specific literature and unified parsing frameworks (Section 2.2), implying that sub-task performance consolidation is better served by existing task-specific surveys.

Limitation 5: The Survey Cannot Distinguish the Effect of Architectural Paradigm from Training Data and Scale — The Causal Claim That Multi-Stage Decomposition Is Superior Is Confounded

The assumption or constraint. The paper's central empirical claim — that multi-stage specialized VLMs outperform end-to-end specialized VLMs on structure-sensitive document parsing — is supported by the ranking in Table 2, where GLM-OCR (0.9B, multi-stage) achieves 94.62 Overall versus FireRed-OCR (2B, end-to-end) at 92.94. The claim is framed in the paper as an architectural finding: the advantage comes from "controlled decomposition" that separates layout analysis from region-level recognition, improving efficiency and reducing hallucination. However, the compared models differ across multiple dimensions beyond architecture — training data composition, data volume, pretraining strategy, optimization procedure, and output format — and none of these are controlled.

The consequence. The observed performance gap cannot be causally attributed to the multi-stage architecture. GLM-OCR trains on 200M "application-aligned pretraining samples" (Section 9.3). FireRed-OCR uses "geometry-aware data generation and a three-stage training curriculum combining multi-task alignment, structured SFT, and format-constrained GRPO optimization" (Section 9.2). PaddleOCR-VL-1.5 uses the Real5-OmniDocBench with specifically designed robustness training. These data and training differences could be the primary drivers of performance, with architecture playing a secondary or negligible role. Alternatively, the architectural advantage might be real but smaller than Table 2 suggests, with the training data quality advantage inflating the gap. Without controlled experiments — training end-to-end and multi-stage variants on identical data with identical optimization — the survey cannot distinguish these possibilities.

This confounding is particularly acute because training data is often the proprietary advantage of the systems being compared. GLM-OCR's "200M application-aligned pretraining samples" presumably include high-quality document-annotation pairs that encode exactly the structural knowledge that general VLMs lack. If a general-purpose VLM like Qwen3-VL-235B were fine-tuned on the same 200M samples, would it close the gap with GLM-OCR? The survey provides no evidence either way because such an experiment has not been conducted (or at least not published and included).

What evidence exists in the paper. The paper acknowledges this implicitly when describing the models: each model's description (Sections 9.1-9.3) emphasizes unique training data and optimization strategies as key differentiators. GLM-OCR is described as integrating "a 0.4B-parameter CogViT visual encoder with a 0.5B-parameter GLM language decoder" with "Multi-Token Prediction mechanism to accelerate decoding" — multiple innovations beyond just the multi-stage architecture. FireRed-OCR's "progressive training framework" is described as its key innovation, not its end-to-end nature. The survey catalogs these differences but does not disentangle them. The one model family that could partially control for architecture is MonkeyOCR (multi-stage, SRR paradigm) and its variants (MonkeyOCR-pro, MonkeyOCR-3B), but there is no corresponding end-to-end model trained on identical data.

Mitigation status. The paper does not address this confounding. It treats the architectural categories as the primary analytical lens (Section 3.3) and presents Table 2 as evidence for architectural differences, without discussing alternative explanations for the observed performance gaps. The limitation is inherent to survey methodology — the survey can only analyze what the literature contains, and the literature does not contain controlled architectural comparisons. Acknowledging this limitation would not diminish the survey's value (the taxonomy remains useful even if the causal claims are uncertain), but the current presentation implies stronger architectural conclusions than the evidence supports.

Limitation 6: Robustness to Real-World Distortions Is Asserted as Critical but Never Measured — The Survey Cannot Characterize Performance Under Deployment Conditions

The assumption or constraint. The survey repeatedly emphasizes the importance of robustness to real-world document conditions — "blurred, skewed, watermarked, or folded document images" (Section 10.2), "handwritten documents, historical archives, and low-resource languages" (Section 10.2), and "robustness to physical artifacts as a critical yet underexplored dimension in purely generative models" (Section 9.3). It catalogs benchmarks designed to evaluate robustness: Real5-OmniDocBench with "five realistic disturbances — scanning, warping, screen photography, illumination variation, and skew" (Appendix I.2), OceanOCR which "emphasizes real-world conditions such as dense bilingual documents and handwriting" (Appendix I.2), and DocPTBench which evaluates "photographed document parsing and translation" (Appendix I.2). However, the survey does not report any quantitative results from these benchmarks for any of the models discussed.

The consequence. The reader has no empirical basis for assessing which paradigms or models are more robust under real-world conditions. The paper asserts that multi-stage VLMs improve robustness — "PaddleOCR-VL and its v1.5 upgrade introduce PP-DocLayoutV3, which replaces rectangular bounding boxes with multi-point representations to model non-planar distortions such as warping and skewing" (Section 9.3), and that this "better reflects deployment conditions in scanned and photographed documents than benchmarks centered on clean rendered pages" — but provides no numbers showing reduced degradation under distortion. Conversely, the paper identifies end-to-end models' reading-order inconsistencies as a failure mode in complex layouts, but does not show whether these inconsistencies are exacerbated or mitigated under physical noise.

This is a significant practical gap because deployed document parsing systems rarely encounter the clean, well-formatted documents that dominate academic benchmarks. A system that achieves 94% Overall on OmniDocBench-v1.5 (clean documents) might drop to 60% on photographed receipts with uneven lighting, and neither the survey nor the underlying literature provides this characterization. The choice between paradigms for a practical deployment — where robustness may be more important than clean-document accuracy — is therefore uninformed by the survey's quantitative evidence.

What evidence exists in the paper. The survey describes robustness-oriented benchmarks (Appendix I.2) and mentions that Real5-OmniDocBench reveals "substantial degradation of both specialized systems and MLLMs under real capture conditions" (Appendix I.2), quoting the DocPTBench paper's finding. But this degradation is never quantified: we do not know whether the degradation is 5% or 50%, whether multi-stage models degrade less than end-to-end models, or whether certain perturbation types (e.g., skew versus illumination) differentially affect architectural paradigms. The paper's own statement that "robustness to physical distortions nevertheless remains uneven across element types and output formats" (Section 9.3) acknowledges the gap without filling it.

Mitigation status. The paper identifies robustness evaluation as a key future direction: "Research should pivot toward real-world challenges, such as blurred, skewed, watermarked, or folded document images" (Section 10.2). It also advocates for "evaluation schemes that reflect quality in actual industrial application scenarios" as a shared priority for academia and industry. But it does not provide such evaluation itself, and its primary empirical contribution (Table 2) uses a benchmark (OmniDocBench-v1.5) that, while more diverse than earlier benchmarks, is still weighted toward clean rendered and scanned documents rather than in-the-wild captures. The survey's value here is in identifying the gap and cataloging the relevant benchmarks — future work, rather than this paper, must fill it.

7. Implications and Future Directions

How This Work Changes the Landscape

This survey does not introduce a new parsing method or advance a single benchmark score. It changes the landscape by providing the first unified taxonomic and empirical scaffolding for a field that has been growing in three disconnected silos — task-specific pipelines, end-to-end VLM-based parsers, and multi-stage hybrid systems — without a shared vocabulary for comparing them or understanding their relationship. The magnitude of this contribution is a conceptual reframing, not a paradigm shift: it does not replace existing approaches, but it enables the community to reason about them systematically for the first time.

The most consequential reframing is the architectural axis as the organizing principle of document parsing research (Section 3.3). Prior to this survey, the implicit narrative in the literature was progressive: rule-based methods → deep learning pipelines → end-to-end VLMs, with each stage rendering the previous one obsolete. The survey dismantles this narrative by showing that multi-stage specialized VLMs — which reintroduce pipeline-style decomposition within unified neural architectures — achieve the highest benchmark performance on structure-sensitive metrics while using 60–400× fewer parameters than general-purpose VLMs (Table 2: GLM-OCR-0.9B at 94.62 Overall vs. Qwen3.5-397B at 90.80). This is not a minor benchmark detail; it overturns the assumption that end-to-end generation represents the destination toward which the field is converging. The data demonstrate that controlled structural decomposition is a competitive design philosophy with its own performance advantages, not a transitional compromise. The multi-stage vs. end-to-end distinction within VLMs — a novel conceptual contribution of this survey — provides the vocabulary for this reframing.

A second shift is the consolidation of cross-paradigm performance data. Table 2 brings 30 models from three architectural categories onto a single benchmark (OmniDocBench-v1.5) with six metrics, enabling the first direct comparison of general-purpose VLMs, end-to-end specialized models, and multi-stage specialized models. This is not merely convenient; it reveals patterns that were invisible when each model was evaluated in isolation. The largest performance gaps between paradigms appear on table structure recognition (TableTEDS: GLM-OCR at 93.96 vs. Gemini-3.0 Pro at 88.28, a 5.68-point gap) and formula recognition (FormulaCDM: PaddleOCR-VL-1.5 at 94.21 vs. Gemini-3.0 Pro at 89.18, a 5.03-point gap) — sub-tasks where explicit structural modeling provides the greatest benefit. The gap on pure text recognition (TextEdit) is narrower but still favors specialized models (GLM-OCR at 0.030 vs. Gemini-3.0 Pro at 0.065). This metric-specific stratification tells researchers where to invest effort: table and formula recognition are the sub-tasks where architectural decomposition yields the largest returns, while text recognition is becoming commoditized across paradigms.

The survey also reconciles a latent contradiction in the document parsing literature: the tension between papers arguing that VLMs make modular pipelines obsolete (exemplified by the "General OCR Theory" framing of GOT-OCR2.0, which advocates a unified end-to-end model for all artificial optical signals) and papers that continue to improve individual pipeline components (new layout analysis methods, specialized formula recognizers, table structure extractors). The survey's taxonomy resolves this by showing that both perspectives are partially correct: VLM-based systems do outperform traditional pipelines on holistic parsing benchmarks, but the best-performing VLMs are precisely those that reintroduce pipeline-style decomposition. Improving individual components remains valuable not because they will be deployed as standalone systems, but because their quality determines the quality of training data and intermediate representations within multi-stage VLM architectures. The paper makes this explicit in Section 10.1: "enhancing the parsing performance of individual document elements remains indispensable... improving these sub-modules not only provides cost-effective and high-quality solutions for niche applications but also generates high-fidelity synthetic data for training unified models."

A diagnostic shift emerges from the survey's cross-paradigm analysis of failure modes. By documenting that both pipeline systems (through error cascading) and end-to-end VLMs (through hallucination and reading-order inconsistencies) fail in structurally similar ways — producing outputs that are locally plausible but globally incorrect — the paper identifies structural verification as the unifying bottleneck across paradigms (Section 10.1). This redirects research attention: rather than treating pipeline errors and VLM hallucinations as separate problems requiring separate solutions, the field should invest in developing robust verifiers and structural consistency rewards that can improve both paradigms. The recent trend toward reinforcement learning in VLM training — Logics-Parsing's layout-aware rewards, olmOCR 2.0's RLVR with binary unit tests, OCRVerse's domain-specific RL rewards — can be understood as the VLM community independently discovering the same solution that pipeline systems access through explicit PRMs. The survey makes this connection visible for the first time.

The survey makes some research directions less attractive by clarifying where diminishing returns have set in or where the problem is fundamentally limited by current paradigms:

Purely sequence-based formula recognition without structural modeling (Section 6.2.1): The paper documents that these methods "still linearize inherently two-dimensional structures, which limits their ability to represent hierarchical relations explicitly," and the strong performance of structure-aware models (TAMER, SSAN, SCAN) and multi-stage VLMs on FormulaCDM (PaddleOCR-VL-1.5 at 94.21) suggests the frontier lies in explicit structural reasoning, not better sequence decoders.
Applying ever-larger general-purpose VLMs without task-specific adaptation (Section 9.1 vs. Table 2): The 397B Qwen3.5 achieves 90.80 Overall vs. GLM-OCR-0.9B at 94.62 — a 3.82-point deficit despite ~440× more parameters. This is strong evidence that scaling general-purpose pretraining alone will not solve document parsing; task-aligned architecture and training are more effective.
Standalone table detection research (Section 7.1): The paper acknowledges that table detection has "achieved relatively mature performance on many benchmarks, and research efforts have increasingly shifted toward more challenging tasks such as table structure understanding." This signals that detection-focused work without structure recognition is approaching ceiling returns.

Conversely, the survey makes several directions more attractive:

Multi-stage architectures with explicit layout-reasoning decomposition as a primary design paradigm (Section 9.3), validated by Table 2's empirical ranking.
Reinforcement learning with structural rewards as a post-training strategy for both end-to-end and multi-stage VLMs (Section 9.2), motivated by the verifier over-optimization diagnosis.
Robustness to physical document distortions as an evaluation and training priority (Section 9.3, Section 10.2), motivated by the acknowledged gap between clean-benchmark performance and real-world degradation.

Follow-Up Research This Work Enables

Controlled cross-paradigm training experiment with identical data. The survey's central empirical finding — that multi-stage models outperform end-to-end models on structure-sensitive metrics — is confounded by training data differences. GLM-OCR trains on 200M application-aligned samples; FireRed-OCR uses geometry-aware data generation with a three-stage curriculum. A critical follow-up would train multi-stage and end-to-end variants on identical document-annotation pairs, controlling for data volume, data quality, and optimization budget, then evaluate on multiple benchmarks (OmniDocBench-v1.5, CC-OCR, olmOCR-Bench). If the multi-stage advantage persists under controlled conditions, it establishes architectural decomposition as causally beneficial rather than merely correlated with superior training data. If it disappears, the field's attention should shift from architecture to data curation strategies. The survey's taxonomy provides the experimental design vocabulary: fix data, vary architecture along the end-to-end vs. multi-stage axis.

Inference cost-normalized benchmarking with latency and FLOPs reporting. The paper identifies deployment cost as a critical unmeasured variable (Section 10.1) and documents that parameter count is an incomplete proxy. A concrete follow-up would evaluate the top 5-10 models from Table 2 on standardized hardware, reporting: inference FLOPs per page, wall-clock latency at batch size 1, throughput at maximum batch size, and GPU memory footprint. Results should be stratified by document page density (number of layout elements) to test whether multi-stage models' per-element processing causes latency to scale with page complexity — a hypothesis the survey's architectural analysis implies but never tests. If a 0.9B multi-stage model requires more total FLOPs than a 3B end-to-end model on dense pages (due to multiple forward passes), the efficiency narrative shifts from "smaller model wins" to "architectural choice depends on page density."

Robustness-stratified evaluation across five perturbation types. Real5-OmniDocBench exists but is reported in Table S1 without quantitative results. A direct follow-up would evaluate the top models from each VLM category (Gemini-3.0 Pro for general-purpose, FireRed-OCR for end-to-end specialized, GLM-OCR for multi-stage specialized) on all five perturbation types: scanning artifacts, geometric warping, screen photography (Moiré patterns), illumination variation, and skew. The key questions: Does the multi-stage advantage on clean documents (Table 2) persist under each perturbation? Do certain perturbations differentially affect paradigms — e.g., does end-to-end generation become more fragile under warping that disrupts raster-order assumptions, while multi-stage layout-then-recognition pipelines remain stable because layout detection operates on downsampled images? The paper's qualitative claim that multi-stage models improve robustness (Section 9.3, discussing PaddleOCR-VL's multi-point bounding boxes) would be tested directly.

Difficulty-stratified parsing accuracy: when does each paradigm fail? The survey's analysis of failure modes (Section 9.1, 9.2) is qualitative — reading-order inconsistencies in multi-column layouts, hallucination in dense scientific PDFs. A quantitative follow-up would partition OmniDocBench-v1.5 (or a composite benchmark) into difficulty strata based on: number of columns, density of floating elements (figures, tables, equations), presence of nested tables, and ratio of non-text elements to text area. Evaluate the top models from each paradigm on each stratum. If end-to-end models degrade sharply on multi-column documents (as the qualitative analysis predicts) while multi-stage models maintain accuracy, the survey's architectural diagnosis is validated quantitatively. If all paradigms degrade similarly, the failure is not architectural but fundamental to current training objectives.

Verifier-guided decoding within VLM architectures. The survey identifies verifier over-optimization as a cross-paradigm bottleneck and documents that pipeline systems use PRM-guided search (beam search with step-level scoring) while VLM systems are beginning to incorporate reinforcement learning with structural rewards. A natural integration would be: use a document-trained PRM (process reward model, trained on step-level correctness of parsing decisions) to guide autoregressive decoding in an end-to-end VLM parser. At each generation step, the PRM scores the partial structured output (e.g., the HTML tree generated so far for a table) and the model uses that score to decide whether to continue, revise, or backtrack — analogous to the lookahead search described in pipeline systems but integrated into VLM token generation. This would test whether explicit structural verification during inference reduces hallucination and improves table structure fidelity beyond what RLVR-based training alone achieves. The experiment would compare: (a) VLM with standard autoregressive decoding, (b) VLM with RLVR training but standard decoding, (c) VLM with RLVR training plus PRM-guided decoding, on table and formula benchmarks.

Principled difficulty estimation for compute-adaptive document parsing. The survey notes that difficulty estimation is expensive (requiring many samples), but the concept — allocating more compute to harder documents — has clear applicability to document parsing. A follow-up would train a lightweight classifier to predict parsing difficulty from document-level features (page density, layout complexity metrics, predicted number of tables/equations) before the full parsing pass, then allocate budget accordingly: easy single-column text documents get a fast lightweight parser, multi-column papers with dense equations get a more expensive multi-stage VLM. The classifier can be trained using the per-page metric scores from a benchmark like OmniDocBench as difficulty labels. The key metric is the accuracy-efficiency trade-off curve: can difficulty-adaptive routing achieve the same average accuracy as always using the heaviest parser, at a fraction of the cost?

Practical Applications and Downstream Use Cases

Training data construction for large language models at scale. The survey documents that structured document representations are "central to building knowledge bases and training corpora for Retrieval-Augmented Generation" (Section 1) and that parsing quality directly determines the quality of knowledge accessible to LLMs. The practical implication of Table 2 is that a 0.9B multi-stage specialized parser (GLM-OCR, 94.62 Overall) can extract higher-fidelity structured content from scientific PDFs, legal documents, and technical reports than general-purpose systems with hundreds of billions of parameters. For an organization constructing a domain-specific knowledge base — extracting all tables, formulas, and structured text from a corpus of 100,000 academic papers — the choice between deploying a 397B general model (90.80 Overall, but enormous inference cost) and a 0.9B specialized model (94.62 Overall, deployable on modest hardware) translates to both higher extraction quality and dramatically lower operational cost. The 3.82-point Overall gap on OmniDocBench-v1.5, concentrated in table and formula metrics (5.68-point gap in TableTEDS), means fewer garbled tables entering the knowledge base and fewer incorrectly transcribed equations polluting training data.

High-fidelity digitization of historical and legal archives. The survey emphasizes robustness to real-world distortions as a critical frontier (Section 10.2), and the qualitative analysis of PaddleOCR-VL's multi-point bounding box representations (Section 9.3) targets exactly the conditions encountered in scanned archival documents: warping from book bindings, skew from imperfect scanning, illumination variation from aged paper. Archives of historical scientific literature, patent filings, or legal contracts cannot be re-scanned under ideal conditions — they exist as-is, with decades of physical degradation. A multi-stage specialized VLM trained with distortion-aware data augmentation (as PaddleOCR-VL-1.5's Real5-OmniDocBench training suggests) can process these documents with substantially higher structural fidelity than general-purpose OCR systems. The specific benefit is not just text extraction but structure preservation: a 1920s chemistry paper with hand-drawn molecular structures, or a patent with nested legal claims in multi-column format, requires both content recognition and structural preservation to be useful for search and analysis. The survey's documentation of OCSR (Section 8.2) and chart parsing advances (Section 8.1) extends this benefit beyond text to domain-specific visual elements.

Cost-efficient batch inference pipelines for document processing platforms. Organizations running large-scale document ingestion — cloud-based document AI platforms, enterprise content management systems, academic search engines — process millions of pages daily. The survey's documentation of the compute-accuracy trade-off across model categories (Table 2) enables an economically informed deployment architecture: use a general-purpose VLM for document classification and routing (coarse-grained tasks where 72B+ models excel), then dispatch to a compact multi-stage specialized parser for structured extraction. The key efficiency insight is not that multi-stage models are more accurate (94.62 vs. 90.80 Overall) but that they achieve higher accuracy with 0.9B parameters that can run on commodity GPUs at high throughput, while the 397B general model requires expensive, scarce hardware and introduces latency that bottlenecks batch pipelines. For a platform processing 10 million pages per month, the difference between deploying 0.9B parsers on T4 GPUs versus 397B models on H100 clusters is the difference between a sustainable service margin and an economically nonviable product, independent of the accuracy advantage.