Qianfan-OCR: A Unified End-to-End Model for Document Intelligence

ArXiv: 2603.13398

🎯 Pitch

A 4B vision-language model matches the best pipeline systems on document parsing benchmarks by letting an optional 'thinking' phase recover explicit layout analysis inside an end-to-end architecture, yet the same layout reasoning that fixes complex, heterogeneous pages harms performance on simple single-column documents—revealing a sharp structural threshold where spatial priors stop helping and start hurting.

1. Executive Summary

This paper introduces Qianfan-OCR, a 4B-parameter unified end-to-end model that integrates document parsing, layout analysis, and document understanding within a single vision-language architecture, evaluated on OmniDocBench v1.5, OlmOCR Bench, and a suite of general OCR and document understanding benchmarks. The core architectural contribution is Layout-as-Thought — an optional thinking phase triggered by <think> tokens where the model generates structured layout representations (bounding boxes, element types, and reading order) before producing final outputs — which bridges the functionality gap between end-to-end models and traditional pipeline systems that natively provide spatial grounding. Qianfan-OCR achieves a 93.12 overall score on OmniDocBench v1.5, ranking first among all end-to-end models and narrowing the gap with the top pipeline system (PaddleOCR-VL 1.5 at 94.50), and attains the highest average score (87.9) across five public Key Information Extraction benchmarks, surpassing Gemini-3.1-Pro, Gemini-3-Pro, Seed-2.0, and Qwen3-VL-235B-A22B. The Layout-as-Thought analysis reveals a sharp boundary condition: the thinking mode provides accuracy gains on structurally complex documents with high layout label entropy, but introduces overhead and degrades performance on homogeneous single-column pages, establishing that explicit layout reasoning benefits spatial grounding only when document heterogeneity exceeds a task-specific threshold.

2. Context and Motivation

The Core Problem: The Three-Way Trade-off in Document Intelligence

The paper addresses a structural tension in the field of optical character recognition and document understanding that the authors explicitly frame as a "three-way trade-off between cost, accuracy, and capability" (Section 1). This is not a single design flaw but rather a systemic fragmentation in how document intelligence systems are built, where practitioners are forced to choose between three mutually incompatible approaches, each with sharp limitations:

• Traditional OCR pipelines (small specialized models chained together) offer low inference cost and high throughput, but require complex multi-stage preprocessing and postprocessing, introduce inter-stage error propagation, and discard spatial-visual context during text extraction.
• Specialized OCR large models improve accuracy through two-stage architectures — layout detection followed by element-wise recognition — but introduce deployment complexity, remain vulnerable to cascading errors across stages, and irreversibly lose visual context when converting document images to text-only representations.
• General vision-language models (Qwen-VL, InternVL, Gemini) offer broad multimodal reasoning but incur higher inference costs, underperform specialized systems on structure-sensitive metrics (table TEDS, reading order accuracy), and lack fine-grained layout control — their OCR capability is a side effect of broad multimodal training, not an optimized first-class capability.

This trade-off creates a practical impasse for industrial deployment. The paper is explicit about the real-world consequences (Section 1):

"In practice, industrial OCR applications — document retrieval with chunking and indexing, contract review, key information extraction from receipts and certificates — often chain detection models, OCR models, and separate LLMs for downstream understanding. This fragmented approach increases deployment cost, limits end-to-end optimization, and requires careful orchestration of heterogeneous components."

The problem, then, is not simply that current systems are imperfect — it is that the dominant architectural paradigms are fundamentally incompatible with the demands of real-world document intelligence, which requires simultaneous precision in structured parsing (preserving layouts, tables, reading order), explicit spatial grounding (bounding boxes, element types), and high-level semantic reasoning (document QA, chart understanding, key information extraction). No single existing system delivers all three.

A Specific, Underappreciated Gap: The Loss of Layout Analysis in End-to-End Models

The paper identifies a subtler but equally important gap that is orthogonal to the cost-accuracy-capability trade-off: the loss of explicit layout analysis functionality in end-to-end OCR models. This is a practical limitation rather than a theoretical one, but it has significant consequences for real-world workflows.

The authors frame it as follows (Section 1):

"A practical limitation of end-to-end OCR is the loss of explicit layout analysis, a capability that pipeline users routinely rely on for element localization and type classification."

Traditional pipeline systems inherently produce layout analysis as a byproduct of their detection-first architecture. When a system like PaddleOCR-VL processes a document, it first runs a dedicated layout detection module that outputs bounding boxes and element type labels (text block, title, figure, table, formula, header, footer) — information that downstream applications consume for tasks like structured extraction, spatial search, accessibility tagging, and document reconstruction. End-to-end models like Nougat, GOT-OCR 2.0, DeepSeek-OCR, and olmOCR skip this step entirely: they directly map document images to structured text (Markdown, LaTeX), producing no intermediate spatial grounding. This means that users who need to know where a particular piece of text appears on the page, or what type of element it belongs to, cannot get this information from an end-to-end model without bolting on a separate detection stage — which defeats the purpose of the unified architecture.

The paper's Layout-as-Thought mechanism is designed specifically to close this gap: it recovers explicit layout analysis within the end-to-end paradigm, making spatial grounding an optional output of the same model rather than requiring a separate detection pipeline.

Why This Problem Matters: Real-World Stakes

The paper's motivation is grounded in concrete deployment scenarios rather than abstract benchmarks. Several factors make this problem pressing:

1. The explosion of document volumes in enterprise settings. Organizations across legal, financial, healthcare, and government sectors process millions of documents daily — contracts, invoices, medical records, regulatory filings, academic papers, technical manuals. These documents are heterogeneous in layout (single-column, multi-column, mixed text-figure-table), language (often multilingual), and format (scanned images, born-digital PDFs, photographs). A system that can handle this diversity with a single model — rather than a fragile chain of task-specific components — represents a step-change in deployment simplicity and maintenance cost.

2. Structured extraction increasingly requires spatial reasoning. Modern document understanding goes beyond "transcribe the text." Applications like contract review need to identify which clauses belong to which sections, table extraction needs to preserve hierarchical structure and merged cells, and chart understanding needs to map data points to labeled axes. All of these tasks require joint visual and textual reasoning that two-stage pipeline systems structurally cannot perform because they discard visual layout during text extraction. The paper provides striking evidence of this in Section 6.3 (Table 6): two-stage OCR+LLM systems achieve zero accuracy on CharXiv (both descriptive and reasoning questions), because chart structures, axis relationships, and data point positions — discarded during text extraction — are essential for answering questions about academic figures.

3. The latency and deployment complexity of pipeline systems is a hidden cost. Multi-stage pipelines involve CPU-based detection modules, GPU-based recognition, and LLM-based understanding, with asynchronous orchestration between stages. Under high concurrency, the CPU-based layout analysis stage becomes a bottleneck that throttles GPU utilization — a problem that worsens with more powerful GPUs. The paper quantifies this in Section 6.5: Qianfan-OCR with W8A8 quantization achieves 1.024 pages per second on a single A100, comparable to the top pipeline system (PaddleOCR-VL at 1.224 PPS), despite having a 4B-parameter language model backbone substantially larger than the detection and recognition modules in pipeline systems. This is possible because end-to-end processing eliminates the CPU bottleneck and enables efficient GPU batching of whole-page images.

4. The training data flywheel for document intelligence is accelerating. The availability of large-scale PDF corpora (arXiv, Common Crawl, web archives) and programmatic data synthesis pipelines means that end-to-end models can be trained on ever-larger and more diverse document datasets. The paper's own data synthesis pipelines (Section 3.2) — spanning document parsing, key information extraction, complex tables, chart understanding, formula recognition, and multilingual OCR across 192 languages — demonstrate that the data barrier for end-to-end document intelligence is rapidly falling. The limiting factor is no longer data availability but architectural design: how to build models that can absorb this diverse supervision signal into a single unified system.

Prior Approaches and Their Shortcomings

The paper situates itself against three established technical routes in OCR (Section 2), each with distinct and well-documented failure modes:

Pipeline OCR Systems

Pipeline systems (PaddleOCR-VL, MonkeyOCR, MinerU 2.5) decompose document parsing into sequential stages: layout detection → element-wise recognition → rule-based assembly. Their architecture is illustrated in Figure 3(a) of the paper.

Strengths: These systems provide explicit layout analysis output (bounding boxes, element types) as a natural byproduct of their detection-first design. They achieve strong accuracy on structured parsing benchmarks through modular optimization — each component can be independently improved. PaddleOCR-VL 1.5 achieves 94.50 on OmniDocBench v1.5, the highest score across all systems.

Critical failures:

• Inter-stage error propagation: If the layout detection module misclassifies a table as a text block, or misses a footnote entirely, the subsequent recognition and assembly stages have no mechanism to recover. The error cascades irreversibly through the pipeline.
• Irreversible loss of visual context: Once text is extracted from document regions, the spatial relationships between elements — which text block is above another, how figures relate to captions, the hierarchical structure of nested tables — are discarded. This is catastrophic for downstream understanding tasks that require spatial reasoning. Table 6 quantifies the damage: two-stage systems built by pairing specialized OCR models with Qwen3-4B achieve 0.0 on CharXiv, 7.0–56.8 on ChartQA (vs. 88.1 for Qianfan-OCR), and 55.9–67.1 on DocVQA (vs. 92.8 for Qianfan-OCR).
• Deployment complexity: As described in Section 6.5, pipeline systems involve heterogeneous stages (CPU-based detection, GPU-based recognition) that require careful orchestration of per-stage concurrency, queue depths, and resource allocation — a significant operational burden.

End-to-End OCR Models

End-to-end approaches (Nougat, GOT-OCR 2.0, DeepSeek-OCR, olmOCR, Dolphin v2, Logics-Parsing) directly map document images to structured outputs using a single vision-language model.

Strengths: These models eliminate inter-stage error propagation and retain full visual context throughout processing. They can be jointly optimized end-to-end on all tasks, and they avoid the deployment complexity of heterogeneous pipeline orchestration. Recent advances — olmOCR 2's GRPO reinforcement learning with unit-test rewards, Dolphin v2's analyze-then-parse strategy — have pushed recognition accuracy into competitive range with pipeline systems.

Critical failures:

• Loss of explicit layout analysis: The paper's central critique is that these models "primarily focus on recognition accuracy or efficiency but lack explicit layout analysis output" (Section 2). They produce structured text but not bounding boxes, element type labels, or reading order — a functionality gap relative to pipeline systems that Qianfan-OCR's Layout-as-Thought is designed to address.
• Unclear scalability ceiling: The paper acknowledges that "as a pioneering attempt at end-to-end OCR, the ultimate performance ceiling of purely end-to-end architectures remains an open question" (Section 7). Pipeline systems benefit from decades of optimization on detection, recognition, and assembly as separate sub-problems; end-to-end models must solve all three simultaneously, and whether they can fully match heavily optimized pipeline systems on the latter's home turf (pure recognition accuracy) is not yet established.
• Training data dependency: End-to-end models typically require large-scale supervised fine-tuning with high-quality layout annotations. The paper relies on this paradigm (SFT with layout annotations from PaddleOCR-VL), and acknowledges that future work could augment it with reinforcement learning.

General Vision-Language Models

Large VLMs (Qwen-VL, InternVL, Gemini) exhibit OCR capabilities as a byproduct of broad multimodal pretraining, but are not optimized for structured document parsing.

Strengths: These models offer strong general reasoning capabilities and are being continuously improved through scaling. Qwen3-VL-4B achieves competitive scores on general OCR benchmarks (873 on OCRBench, 60.68 on OCRBenchv2 English) and document understanding (94.9 on DocVQA, 81.8 on TextVQA).

Critical failures:

• Higher inference cost for structured parsing: Because they are not optimized for document-specific tasks, these models require more computation per page than specialized systems. The paper does not directly quantify this, but the architectural cost is inherent: a general VLM encodes the entire image at high resolution without document-specific optimizations like layout-aware tiling or reading-order attention.
• Lack of fine-grained layout control: General VLMs do not natively output bounding boxes, element types, or reading order — they lack the explicit spatial grounding capabilities that pipeline systems provide and that Layout-as-Thought aims to recover.
• Underperformance on structure-sensitive metrics: Table 4 shows that Qwen3-VL-4B achieves 86.78 on OmniDocBench v1.5 vs. Qianfan-OCR's 93.12 — a gap of 6.34 points — with particularly large differences on table-related metrics (TableTEDs: 79.29 vs. 91.02, TableTEDss: 84.26 vs. 93.85). This confirms the paper's claim that general VLMs are not optimized for the precise structure preservation that document parsing requires.

Conflicting Evidence in the Literature

The paper's motivation is further sharpened by an implicit tension in prior work that it seeks to resolve:

• Pipeline advocates point to the raw accuracy numbers: PaddleOCR-VL 1.5 achieves 94.50 on OmniDocBench v1.5, while the best end-to-end model at the time of writing was DeepSeek-OCR-v2 at 91.09 — a 3.4-point gap. From this perspective, end-to-end models are simply not yet competitive on recognition accuracy.

• End-to-end advocates point to the catastrophic failure of pipeline systems on tasks requiring visual-spatial reasoning: zero accuracy on CharXiv, massive degradation on ChartQA and DocVQA (Table 6). From this perspective, the "accuracy" of pipeline systems is a narrow metric that hides their fundamental inability to perform document understanding once visual context is discarded.

• General VLM advocates point to the continuous improvement of large multimodal models through scaling, suggesting that specialized OCR architectures are a temporary stopgap that will be overtaken by general-purpose models as they grow larger and more capable.

Qianfan-OCR's positioning resolves this tension by arguing that none of these perspectives is complete. The paper demonstrates that an end-to-end architecture can:

1. Narrow the recognition accuracy gap with pipeline systems (93.12 vs. 94.50 on OmniDocBench v1.5 — a 1.38-point difference, down from the previous 3.4-point best)
2. Recover explicit layout analysis through Layout-as-Thought, bridging the functionality gap that previously made end-to-end models a non-starter for applications requiring spatial grounding
3. Match or exceed general VLMs on document understanding tasks while maintaining specialized OCR capabilities — achieving 92.8 on DocVQA (vs. 94.9 for Qwen3-VL-4B) but 93.12 on OmniDocBench (vs. 86.78 for Qwen3-VL-4B)

The key insight is that specialization and generality are not mutually exclusive in a sufficiently well-designed end-to-end architecture. The model's ability to switch between OCR-specialist-level accuracy on structured parsing and competitive performance on document understanding — controlled through prompts — demonstrates that the trade-off between these capabilities is not fundamental but rather a consequence of prior architectural choices.

How This Paper Positions Itself

The paper frames its contribution as addressing the "three-way trade-off" by demonstrating that a single end-to-end architecture can simultaneously deliver:

1. Specialized OCR accuracy competitive with pipeline systems (OmniDocBench 93.12, within 1.4 points of the top pipeline system)
2. Explicit layout analysis through Layout-as-Thought, recovering a capability that pipeline systems natively provide and that previous end-to-end models lack
3. Document understanding and reasoning that matches or approaches general VLMs on comprehension benchmarks, while dramatically outperforming two-stage OCR+LLM pipelines on tasks requiring visual-spatial reasoning

The paper's positioning is not to claim that end-to-end architectures are universally superior — it acknowledges limitations on very hard structured parsing where pipeline systems still lead, and on deployment in resource-constrained environments where the 4B parameter footprint is burdensome — but rather to establish that the theoretical ceiling of end-to-end document intelligence is substantially higher than prior results suggested, and that the gap between end-to-end and pipeline performance is narrowing rapidly enough to make unified architectures a practical choice for production deployment.

The paper also explicitly positions itself as an early exploration rather than a solved problem: Section 7 discusses open questions about the ultimate performance ceiling of end-to-end architectures, the need for more flexible and task-adaptive layout reasoning, and the potential for reinforcement learning to strengthen reasoning capabilities. This intellectual honesty — acknowledging the pioneering nature of the work and its limitations — strengthens rather than weakens the contribution.

3. Technical Approach

3.1 Reader Orientation

Qianfan-OCR is a 4-billion-parameter vision-language model that takes a document image as input and directly produces structured text (Markdown, HTML tables, formatted formulas) along with optional spatial grounding information, all within a single forward pass. The system solves the problem of document intelligence fragmentation — where separate detection, recognition, and understanding models must be chained together — by unifying layout analysis, text recognition, and semantic reasoning into one architecture, while recovering the explicit layout analysis capability that pipeline users rely on through an optional "think before you answer" mechanism called Layout-as-Thought.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components, with information flowing from raw pixels to structured output:

1. Vision Encoder (Qianfan-ViT): Takes a document image at variable resolution (up to 4K) and tiles it into $448 \times 448$ patches using the AnyResolution strategy. Processes each patch through 24 Transformer layers with 1024 hidden dimensions and 16 attention heads, producing 256 visual tokens per tile. With a maximum of 16 tiles, a single document can be represented by up to 4,096 visual tokens — providing sufficient spatial resolution for fine-grained character recognition in dense, small-font documents.

2. Cross-Modal Adapter: A lightweight two-layer MLP with GELU activation that projects visual features from the encoder's 1024-dimensional representation space into the language model's 2560-dimensional embedding space. This component is the only one trained during Stage 1 (adapter-only alignment with 50B tokens at a peak learning rate of $1 \times 10^{-3}$), bridging the modality gap before full-parameter training.

3. Language Model Backbone (Qwen3-4B): A 4.0B-parameter decoder-only Transformer (3.6B non-embedding parameters) with 36 layers, 2560 hidden dimensions, and a 32K native context window extensible to 131K via YaRN. Uses Grouped-Query Attention with 32 query heads and 8 KV heads (reducing KV cache memory by $4\times$), and RMSNorm for layer normalization. This component generates token-by-token, conditioned on the projected visual features and optional thinking tokens.

4. Layout-as-Thought Trigger ( thinking tokens): An optional prefix that activates a structured reasoning phase. When prepended to the query, the model first generates bounding box coordinates, element type labels, and content summaries enclosed in <layout>...</layout> tags, then conditions its final response on this intermediate spatial analysis. When omitted, the model produces output directly without layout reasoning.

5. Coordinate Special Tokens (<COORD_0> through <COORD_999>): A dedicated vocabulary of 1,000 special tokens introduced during Stage 3 continued pretraining that encode normalized bounding box coordinates. Each coordinate consumes a single token rather than requiring multiple digit tokens, reducing the thinking phase output length by approximately 50% and substantially decreasing inference latency for complex documents that can contain 60+ layout elements per page.

The flow of computation at inference time: (a) document image enters the vision encoder and is tiled dynamically; (b) visual tokens from all tiles are concatenated and projected through the adapter into the language model's embedding space; (c) if thinking is activated, the language model autoregressively generates a structured layout analysis — listing elements in reading order with their bounding boxes, types, and content summaries — using the coordinate special tokens to compactly represent spatial positions; (d) conditioned on either the raw visual tokens (no-think mode) or the visual tokens plus the self-generated layout analysis (think mode), the model produces the final structured output in the requested format (Markdown, HTML, or task-specific response).

3.3 Roadmap for the Deep Dive

• First, the data synthesis pipelines (Section 3.2 of the paper), because the model's capabilities are fundamentally shaped by the training data, and the paper invests substantial engineering in six domain-specific synthesis pipelines that collectively produce the largest and most diverse OCR training corpus described in the paper. Understanding the data — what labels exist, how they were generated, what augmentations were applied — is prerequisite to understanding what the model learns and why certain design choices (like the 25-category layout taxonomy) matter.

• Second, the vision encoder and AnyResolution tiling mechanism, because the fundamental challenge of OCR is that documents contain dense, small-font text that requires high-resolution processing, and the dynamic tiling strategy is the architectural component that addresses this challenge.

• Third, the cross-modal adapter and the rationale for adapter-only Stage 1 training, because bridging vision and language modalities is a well-known challenge in VLM training, and the paper's progressive training strategy depends on this component working correctly before full-parameter fine-tuning begins.

• Fourth, the language model backbone (Qwen3-4B) and its efficiency mechanisms (GQA, RMSNorm, coordinate special tokens), because the 4B scale is a deliberate engineering compromise between reasoning capability and deployment practicality, and the coordinate special token system is a non-obvious but impactful efficiency optimization.

• Fifth, the Layout-as-Thought mechanism in complete detail, including the training data construction for layout reasoning, the format and semantics of the intermediate output, how coordinate special tokens enable compact spatial representations, and the two mechanisms by which layout reasoning benefits the final output (element-type-aware formatting and reading-order-guided sequencing).

• Sixth, the four-stage progressive training recipe, because this is where the paper's most significant methodological contribution lies: a carefully orchestrated curriculum that builds capabilities from basic cross-modal alignment through domain-specific specialization to instruction following, with specific data mixtures, learning rates, and token budgets at each stage that were validated through ablation studies.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and engineering paper whose core idea is that a single end-to-end vision-language model, trained with a carefully designed multi-stage curriculum on synthetically generated and augmented document data, can simultaneously deliver specialized OCR accuracy competitive with pipeline systems, explicit layout analysis through an optional thinking mechanism, and document understanding capabilities that dramatically outperform two-stage OCR+LLM pipelines on tasks requiring visual-spatial reasoning. The key architectural innovation is not a new model architecture per se — Qianfan-OCR reuses the multimodal bridging architecture from Qianfan-VL — but rather the combination of (1) large-scale domain-specific data synthesis across six OCR sub-domains, (2) a 25-category fine-grained layout taxonomy, (3) the Layout-as-Thought mechanism with coordinate special tokens, and (4) a four-stage progressive training curriculum validated through ablation studies.

Data Synthesis Pipelines: The Foundation of Model Capability

The paper develops six specialized data synthesis pipelines that collectively produce the training corpus for all four training stages. These pipelines are the most substantial engineering contribution in the paper, and they determine what the model can and cannot do.

Document Parsing Data Synthesis. This pipeline converts document images into structured Markdown using PaddleOCR-VL as the annotation engine. The pipeline proceeds in four phases:

1. Layout Detection: PaddleOCR-VL's detection module processes each document image and outputs bounding boxes with element type labels. All bounding box coordinates are normalized to $[0, 999]$ for resolution invariance — meaning that regardless of the original image dimensions (e.g., $1200 \times 1600$ or $2480 \times 3508$), coordinates are rescaled so that the top-left corner is $(0, 0)$ and the bottom-right corner is $(999, 999)$. This normalization ensures that the model learns to reason about relative spatial positions rather than absolute pixel positions.

2. Content Recognition: PaddleOCR-VL's recognition module extracts text from each detected region. Tables are converted to HTML via an intermediate OTSL (Open Table Structure Language) format, and mathematical formulas are wrapped in $$ blocks for LaTeX rendering.

3. Markdown Assembly: The extracted content is assembled into reading-order-preserving Markdown, with appropriate formatting for headings, paragraphs, lists, and interleaved non-text elements.

4. Quality Filtering and Augmentation: Automatic filtering removes repetitive or extreme-length samples. Image-level augmentations — compression artifacts, horizontal and vertical flipping, Gaussian blur — improve robustness to real-world document scanning quality variations.

The 25-category layout taxonomy is a critical design choice. The paper explicitly compares the label systems of PaddleOCR-VL and MinerU 2.5, and the choice has substantial downstream consequences:

"The main difference lies in body text labels: PaddleOCR-VL provides fine-grained categories (text, vertical_text, paragraph_title, doc_title, abstract, content, reference, reference_content, aside_text), while MinerU 2.5 uses coarser labels (text, title, list, aside_text)."

The taxonomy has four groups totaling 25 categories:

• Text elements (12 labels): text, vertical_text, paragraph_title, doc_title, abstract, content, reference, reference_content, aside_text, and several others that distinguish different roles of body text — a level of granularity that directly supports downstream tasks like structured extraction from academic papers (distinguishing abstracts from content, references from body text) and contract analysis (identifying section headings vs. body text vs. footnotes).

• Headers/Footers (4 labels): header, footer, page_number, and one additional category — enabling clean separation of running content from body text during document reconstruction.

• Figures/Tables (6 labels): image, table, figure, seal, and two other visual element categories — spanning both semantic content (figures, tables) and document-specific artifacts (seals on certificates, stamps on forms).

• Formulas (3 labels): display_formula, inline_formula, and one additional category — distinguishing displayed equations (which should be rendered as block LaTeX) from inline mathematical notation (which should remain in text flow).

The paper evaluates both PaddleOCR-VL and MinerU 2.5 on a multi-type document layout benchmark and finds that PaddleOCR-VL "achieves consistently higher detection accuracy." The finer granularity is not merely aesthetic — it enables the model to learn to apply different output formats to different element types during the Layout-as-Thought phase, which the paper explicitly identifies as one of the two mechanisms by which layout reasoning benefits the final response.

Layout-as-Thought Data Construction. This is a distinct data synthesis sub-pipeline that produces the supervision signal for the model's explicit layout reasoning capability:

"We construct training data where the model generates structured layout analysis within thinking tokens before final output, listing bbox coordinates, element labels, and content summaries as intermediate reasoning enclosed in <layout>...</layout> tags."

Each training example in this pipeline consists of a document image paired with a structured trace that includes:

1. Bounding Box Coordinates: Each layout element gets a normalized bounding box specified using the coordinate special tokens, e.g., <COORD_779>,<COORD_057>,<COORD_890>,<COORD_071> representing (x1=0.779, y1=0.057, x2=0.890, y2=0.071) in the normalized $[0, 999]$ space. Four coordinates per box: left, top, right, bottom.

2. Element Type Label: One of the 25 categories from the layout taxonomy, specified in a <label> tag.

3. Content Summary: For text-type elements, a <brief> tag containing a concise natural language summary of the element's content. This is not a verbatim transcription but rather a semantic description — "Section title indicating review and organization" rather than the full title text — which the paper argues helps the model focus on structural reasoning rather than premature text generation.

The format, as illustrated in the paper's example (Figure 5), follows this pattern per element:

<box>[[<COORD_X1>,<COORD_Y1>,<COORD_X2>,<COORD_Y2>]]</box>
<label>element_type</label>
<brief>concise content description</brief>

Elements are enumerated in reading order — the sequential order in which a human would read the document, accounting for multi-column layouts, interleaved figures, and footnotes. This reading-order sequencing is critical: it provides an explicit ordering signal that the response generation phase can follow to produce correctly sequenced output, which the paper identifies as the second of two mechanisms by which layout reasoning benefits the final response.

Key Information Extraction (KIE) Data Synthesis. The KIE data covers two extraction scenarios: complete extraction (extracting all structured fields from a document) and targeted extraction (extracting only user-specified fields given a query). The pipeline incorporates several sophisticated quality control mechanisms:

"To address hallucination in teacher models, we combine open-source data with small model pre-annotations for multi-model collaborative labeling."

The multi-model collaborative labeling approach means that instead of trusting a single teacher model (which might hallucinate field values), the pipeline runs multiple smaller models on the same document and uses agreement between them as a confidence signal. Disagreements are flagged for manual review.

"We implement semantic generalization for keys across different regions and formats, constructing multiple synonymous descriptions for the same field."

Semantic generalization means that the same logical field — say, "total amount" on an invoice — is represented in training data with diverse surface forms ("total," "amount due," "grand total," "payable sum") across different document templates. This prevents the model from overfitting to specific field name strings and improves generalization to unseen document formats.

The pipeline includes hard rule filtering using business logic — the paper gives the example of verifying "unit price × quantity = total" — which catches hallucinated extractions that are internally inconsistent. Difficult sample mining targets long sequences with 5+ detail rows and dense text documents, and sample distribution is rebalanced based on task difficulty to enhance stability in extreme scenarios.

Complex Tables Synthesis. The paper's approach to table data has two complementary streams:

Programmatic synthesis randomly generates tables with 3–20 rows and columns, supporting random cell merging (spanning rows and columns). Content is populated via the Faker library (for realistic dummy data like names, addresses, dates, monetary amounts) or LLMs (for more diverse and coherent textual content), covering diverse data types. The pipeline randomly samples from 50+ professional CSS themes, renders tables via Jinja2 templating and KaTeX (for mathematical content in table cells), and applies geometric transformations, color perturbations, and blur augmentations to simulate real-world scanning conditions. This stream produces tables that are structurally diverse — the random generation of merge patterns is critical because merged cells are a major failure mode for OCR systems — but visually clean.

Real document table extraction complements the synthetic stream by capturing the noise characteristics of actual scanned tables: irregular borders, misaligned text, low contrast, and background artifacts that synthetic pipelines struggle to simulate. The pipeline uses internal parsing tools to detect and extract table regions from real documents, parses them with both PaddleOCR-VL and internal table models, converts both outputs to HTML representation, and performs consistency validation:

"Convert both outputs to HTML, and perform consistency validation to filter samples with significant structural or content differences, ensuring reliable annotations while preserving real document layout and noise characteristics."

The consistency validation is crucial: if the two independent parsers disagree on table structure (e.g., one detects a merged cell where the other detects separate cells), the sample is either discarded or flagged for manual annotation. Only when both parsers agree — providing a form of cross-model verification — is the sample included in training. This dual-source strategy combines the structural diversity of programmatic synthesis with the visual realism of real document tables.

Chart Understanding Synthesis. This pipeline synthesizes over 300,000 chart understanding training examples from arXiv LaTeX sources (2022–present). The methodology is:

1. Source extraction: Figure code blocks are systematically extracted from LaTeX source files through rule-based parsing. This ensures the pipeline captures the exact plotting code that generated each figure, not just an image of the result.

2. Re-rendering: Extracted code is re-rendered using the TexLive engine to obtain lossless vector images, avoiding the JPEG compression artifacts present in arXiv PDFs.

3. Metadata extraction: Caption parameters (chart type, axis labels, data series names) are parsed from the caption text and used as ground truth metadata.

4. Visual description generation: VLMs (separate from the model being trained) generate detailed natural language descriptions of each chart, capturing "visual encoding, statistical features, spatial layout, and fine-grained distribution characteristics." The paper does not specify which VLM is used for description generation.

5. Task-specific reasoning: The pipeline designs custom reasoning tasks for 11 chart categories:

   • Line charts: Trend analysis (identifying inflection points, monotonic regions, rates of change)
   • Scatter plots: Correlation analysis (direction, strength, clustering), distribution characteristics
   • Box plots: Outlier detection, quartile analysis, range comparisons between groups
   • Bar charts: Relative magnitude comparisons, ranking, ratio estimation
   • Pie charts: Proportion estimation, majority/minority identification
   • And six additional chart types with corresponding reasoning tasks

The key insight is that chart understanding requires both visual perception (reading axis labels, identifying data markers, distinguishing color-coded series) and quantitative reasoning (estimating values, comparing magnitudes, detecting trends), and that training data must provide supervision for both simultaneously. The "metadata + visual description" structure ensures that each training example teaches the model to ground its reasoning in both the visual encoding (the chart image) and the underlying data semantics (the caption and axis information).

Multilingual OCR Data Construction. This pipeline extends language coverage to 192 languages using a reverse synthesis approach:

"We adopt a reverse synthesis approach starting from the HPLT multilingual corpus."

Reverse synthesis means that instead of starting from document images and extracting text (the typical OCR direction), the pipeline starts from clean text corpora and renders document images — a text-to-image direction that gives perfect ground truth for free. The HPLT (High Performance Language Technologies) corpus provides text in 192 languages drawn from web crawls.

The pipeline includes writing-system-specific handling:

• Latin/Cyrillic: Standard left-to-right rendering with European typeface selections
• Arabic: Automatic right-to-left text direction detection, Arabic character reshaping (where characters take different forms depending on their position in a word — isolated, initial, medial, or final), and appropriate typeface selection
• South Asian scripts (Devanagari, Tamil, Telugu, etc.): Complex text layout with conjunct formation and matra positioning
• Southeast Asian scripts (Thai, Khmer, Myanmar): Stacking diacritics and tone marks
• Han characters (Chinese, Japanese, Korean): Character-level line breaking with appropriate font selection

Text-font renderability filtering uses the fonttools library to validate that each selected font can actually render each character in the text — preventing training data where the model sees placeholder glyphs (□) instead of actual characters, which would teach the model the wrong mapping.

Diverse typesetting variations — font size (simulating different scanning resolutions), column layout (single, double, triple), margins, line spacing, and texture backgrounds — are randomized to approximate real document distributions. The paper explicitly notes that these variations are not independent random perturbations but are sampled from parameter ranges that reflect real-world document diversity: font sizes corresponding to common print sizes (8pt–14pt), margins typical of A4/letter paper, and background textures drawn from scans of real paper stocks.

Document Image Augmentation. The paper employs two separate augmentation pipelines because different tasks require different invariance properties:

"We employ two augmentation pipelines: one for OCR tasks (allowing mild geometric perturbations) and one for layout parsing tasks (preserving geometric consistency)."

For OCR tasks (where the goal is text recognition), mild geometric perturbations are acceptable because they do not change the underlying text content — a slightly rotated "invoice" still reads as "invoice." The pipeline allows:

• Text noise: Broken strokes (simulating worn typewriter ribbons or low toner), ink bleeding (simulating inkjet printing on absorbent paper), character misalignment (simulating typewriter registration errors)
• Background noise: Texture patterns (simulating paper grain), color drift (simulating aging/yellowing), watermarks
• Imaging noise: Gaussian blur (simulating out-of-focus capture), moiré patterns (simulating photographing printed halftone images), shadow gradients (simulating uneven lighting in camera-based scanning), exposure variation (simulating over/under-exposure), JPEG compression artifacts

For layout parsing tasks (where the goal is element localization and type classification), geometric consistency must be preserved because rotation, perspective distortion, or cropping would change bounding box coordinates, invalidating the layout annotations. The pipeline applies the same noise categories but restricts geometric transformations to only those that preserve the alignment between image pixels and layout bounding boxes.

Rotation augmentation is treated separately and given particular emphasis:

"Rotation augmentation (90°, 180°, 270°, and ±15°) significantly improves performance on KIE and table recognition tasks where documents frequently appear in non-standard orientations."

The four orthogonal rotations (90°, 180°, 270°) cover the common case of documents scanned or photographed in portrait vs. landscape orientation or upside-down. The ±15° range covers small off-angle rotations that frequently occur when documents are photographed by hand rather than placed on a flatbed scanner. The paper notes that rotation augmentation is specifically important for KIE and table recognition — tasks where content must be extracted in a semantically meaningful order, and where misoriented text would break the reading-order logic.

Vision Encoder: AnyResolution Tiling for High-Resolution Document Processing

The vision encoder is based on Qianfan-ViT, pretrained as part of the Qianfan-VL framework, and its design is governed by a fundamental constraint of OCR: documents contain text at highly variable scales and densities, and standard vision encoders that operate on fixed-size inputs (e.g., $224 \times 224$ or $336 \times 336$) do not provide sufficient spatial resolution to distinguish small-font characters, especially in languages with complex character shapes (Chinese, Japanese, Korean, Arabic).

The AnyResolution strategy addresses this by dynamically tiling input images:

"The AnyResolution design dynamically tiles input images into $448 \times 448$ patches, supporting variable-resolution inputs up to 4K."

The tiling mechanism works as follows:

1. Input assessment: The input document image (which may be any resolution, e.g., $1200 \times 1800$ for a scanned letter or $2480 \times 3508$ for an A4 document at 300 DPI) is divided into the minimum number of $448 \times 448$ patches needed to cover the entire image. The tile size was chosen based on the pretrained ViT's patch size of $14 \times 14$, giving $32 \times 32 = 1024$ patches per tile, each producing a visual token.

2. Per-tile encoding: Each $448 \times 448$ patch is independently encoded by the ViT. With a $14 \times 14$ patch size, each tile is split into $32 \times 32$ patches, each represented by a single visual token after the Transformer layers, yielding 256 visual tokens per tile. The ViT consists of 24 Transformer layers with 1024 hidden dimensions and 16 attention heads.

3. Token concatenation: Visual tokens from all tiles are concatenated into a flat sequence. With a maximum of 16 tiles ($4 \times 4$ grid), the encoder produces up to $16 \times 256 = 4,096$ visual tokens per document. This provides substantially more spatial resolution than standard single-tile encoding — a $448 \times 448$ single-tile representation would have only 256 tokens to represent an entire A4 page, while a $4 \times 4$ grid at maximum resolution has 4,096 tokens.

4. Dynamic tile count: The number of tiles is not fixed at 16 but scales with document resolution. A low-resolution scan ($600 \times 800$) might use only $2 \times 2 = 4$ tiles (1,024 visual tokens), while a high-resolution scan ($3,500 \times 5,000$) approaches or reaches the 16-tile maximum. The paper does not specify the exact downsampling logic — whether images are resized to fit within 16 tiles or whether higher-resolution images are cropped — but the intent is clear: the model should use more visual tokens for documents that need more spatial detail.

Why AnyResolution matters for OCR. Standard vision encoders in VLMs typically resize input images to a fixed resolution (e.g., $224 \times 224$ for CLIP-based encoders, or $336 \times 336$ for LLaVA-style models). For document OCR, this is catastrophically lossy. Consider a scanned A4 document at 300 DPI ($2480 \times 3508$ pixels) with 10-point text. Resizing to $336 \times 336$ reduces linear resolution by more than $7\times$, making small text unreadable even in principle. The $448 \times 448$ tile size, combined with up to 16 tiles, provides up to $4 \times 4 \times 448 = 1792$ pixels per dimension — still below the original 300 DPI resolution, but within the range where text remains decipherable, especially with the ViT's per-patch attention mechanism.

The paper does not explicitly compare different tile sizes or tile counts; the $448 \times 448$ and 16-tile maximum appear to be inherited from the Qianfan-VL architecture. However, the rationale is implicitly clear from the OCR context: $448 \times 448$ provides a linear resolution roughly $2\times$ higher than the standard $224 \times 224$ used in many VLMs, and the multi-tile design enables the model to process documents at near-native resolution when needed.

A subtle point not discussed in the paper: the tiling strategy introduces a potential issue with document elements that span tile boundaries. If a text line, table, or figure extends across the boundary between two adjacent tiles, the ViT encodes each half independently, and the model must learn to associate visual features from two different tiles as belonging to the same document element. This is a non-trivial cross-tile reasoning challenge that the paper does not explicitly address, though the AnyResolution training in Qianfan-VL presumably teaches the model this capability through exposure to documents with cross-tile element spans.

Cross-Modal Adapter: Bridging Vision and Language Modalities

The adapter is a lightweight component that solves a well-known problem in vision-language model training: the vision encoder and language model operate in different representational spaces, and directly connecting them without adaptation leads to training instability and slow convergence.

"A lightweight two-layer MLP with GELU activation bridges the vision encoder and the language model, projecting visual features from the encoder's representation space (1024 dimensions) into the language model's embedding space (2560 dimensions)."

The adapter's design is intentionally simple:

• Architecture: Two fully-connected layers with a GELU (Gaussian Error Linear Unit) nonlinearity between them. GELU is chosen over ReLU because it provides smoother gradients — the GELU activation is $\text{GELU}(x) = x \cdot \Phi(x)$ where $\Phi$ is the standard normal CDF, which unlike ReLU is differentiable everywhere and provides a probabilistic "gating" behavior that is beneficial for cross-modal alignment where many visual features should be partially rather than fully suppressed.

• Dimensionality transformation: Input dimension 1,024 (matching the ViT's hidden size), hidden dimension (not specified in the paper, but presumably in the range 1,536–2,560), output dimension 2,560 (matching Qwen3-4B's embedding dimension). The projection is necessary because the ViT and the language model were pretrained independently and use different dimensionalities; the adapter is the only component that bridges these incompatible spaces.

• Parameter count: The paper describes it as "lightweight" and explicitly notes it "minimizes adapter parameters while ensuring effective cross-modal alignment," but does not specify the exact parameter count. Given the two-layer design and the dimensionality transformation, the adapter likely contains approximately $1024 \times h + h \times 2560$ parameters (plus biases), where $h$ is the hidden dimension — putting it roughly in the range of 5–10 million parameters, or about 0.1–0.25% of the total 4B parameter model.

Stage 1 training rationale. The adapter is the only component trained during Stage 1 (50B tokens, peak learning rate $1 \times 10^{-3}$):

"During Stage 1 training, only the adapter is trained with a higher learning rate for fast alignment, while subsequent stages perform full-parameter training."

This adapter-only approach serves two purposes:

1. Stable initialization: Training the adapter first with a high learning rate ($1 \times 10^{-3}$) allows the cross-modal mapping to converge quickly without disturbing the pretrained weights of the ViT (which has learned general visual representations) or the language model (which has learned language understanding and generation). If full-parameter training were attempted from the start, the gradient signals from the randomly initialized adapter would propagate back into the pretrained components, potentially degrading their representations.

2. Warm-start for later stages: Once the adapter produces reasonable projections of visual features into language space, the subsequent full-parameter stages start from a position where the visual and textual representations are already approximately aligned. This reduces the risk of training divergence and allows lower learning rates ($2 \times 10^{-5}$ to $1 \times 10^{-5}$) that fine-tune rather than overwrite pretrained knowledge.

The paper does not provide ablation results on adapter architecture choices (e.g., one-layer vs. two-layer, alternative activation functions, alternative dimensionality reduction strategies like Q-former or perceiver resampling), likely because the adapter design was validated in the original Qianfan-VL work and not re-ablated for this OCR-specific application.

Language Model Backbone: Qwen3-4B with Efficiency Optimizations

The language model backbone is Qwen3-4B, a 4.0B-parameter decoder-only Transformer with 3.6B non-embedding parameters. The paper's choice of this backbone reflects a deliberate engineering trade-off:

"This scale strikes a balance between reasoning capability and deployment efficiency — large enough for complex document understanding and layout reasoning, yet practical for single-GPU serving in production."

Architecture specifications:

• 36 layers, 2560 hidden dimensions: The layer count and hidden size place this model in the "small but capable" regime. For comparison, LLaMA-7B has 32 layers and 4096 hidden dimensions — Qwen3-4B has more layers but narrower, trading depth for width, which can be beneficial for sequential reasoning tasks (like reading a document page in order) where deep representations help with long-range dependencies.

• 32K native context window, extensible to 131K via YaRN: The native 32K context is critical for OCR: a single document page with layout analysis can easily require 2,000–4,000 visual tokens plus the autoregressively generated layout and response tokens. Multi-page documents would exceed 32K, hence the YaRN (Yet another RoPE extensioN) extension to 131K. YaRN is a method for extending the context window of RoPE-based models by interpolating the rotary position embeddings, allowing the model to process sequences longer than those seen during pretraining without catastrophic degradation of attention patterns.

• Grouped-Query Attention (GQA): The model uses 32 query heads but only 8 key-value heads. This means each KV head is shared among 4 query heads.

  The GQA mechanism works as follows: in standard multi-head attention, each attention head has its own query (Q), key (K), and value (V) projections, producing head-specific attention patterns. In GQA, the query projections remain per-head (32 sets), but the key and value projections are shared across groups of heads (8 sets, each serving 4 query heads). The attention computation for a single group is:

  $\text{Attention}(Q_1, Q_2, Q_3, Q_4, K_g, V_g) = \left[\text{softmax}\left(\frac{Q_1 K_g^T}{\sqrt{d_k}}\right)V_g, \text{softmax}\left(\frac{Q_2 K_g^T}{\sqrt{d_k}}\right)V_g, \ldots\right]$

  where $Q_1, Q_2, Q_3, Q_4$ are the query projections for the four heads in the group, $K_g$ and $V_g$ are the shared key and value projections for the group, and $d_k$ is the per-head dimension. The softmax is computed separately for each query head against the shared key, and each query head attends to the shared value with its own attention weights. The outputs are concatenated along the head dimension.

  What this achieves: The KV cache — which stores the key and value tensors for all previously generated tokens to avoid recomputing them — is reduced by a factor of 4 ($32/8$) compared to standard multi-head attention with 32 KV heads. For a 32K context window, the KV cache memory for standard MHA would be (approximately) $32 \text{ heads} \times 32\text{K tokens} \times 80 \text{ dim} \times 2 \text{ (K+V)} \times 2 \text{ bytes (FP16)} \approx 328 \text{ MB}$; with GQA, this drops to $8 \text{ heads} \times 32\text{K tokens} \times 80 \text{ dim} \times 2 \times 2 \approx 82 \text{ MB}$. This reduction is critical for single-GPU serving, where KV cache memory competes with model parameters and batch activations for limited VRAM.

  Why GQA works for OCR: The paper cites Ainslie et al. (2023), which showed that GQA maintains generation quality because the key-value sharing does not significantly constrain the attention patterns — the diversity of attention comes primarily from the distinct query projections, and sharing keys and values across a small group of heads still allows each head to focus on different aspects of the input. For document processing specifically, the shared keys and values mean that different attention heads within a group attend to the same context information (the visual tokens representing the document) but through different query projections — analogous to different "questions" being asked about the same document content.

• RMSNorm for layer normalization: The model uses RMSNorm (Root Mean Square Normalization) rather than LayerNorm:

  $\text{RMSNorm}(x) = \frac{x}{\sqrt{\frac{1}{d}\sum_{i=1}^d x_i^2 + \epsilon}} \cdot \gamma$

  where $x$ is the input vector of dimension $d$, $\epsilon$ is a small constant for numerical stability, and $\gamma$ is a learnable scale parameter.

  What this computes: RMSNorm normalizes each vector by its root-mean-square value (the square root of the mean of squared elements), then rescales by a learnable parameter $\gamma$. Unlike LayerNorm, which subtracts the mean before dividing by the standard deviation, RMSNorm only divides by the RMS — there is no mean centering.

  Why RMSNorm over LayerNorm: The paper cites Zhang and Sennrich (2019), who showed that RMSNorm provides training stability comparable to LayerNorm while being computationally cheaper (no mean computation or subtraction) and, critically, that the mean-centering step in LayerNorm may be unnecessary — the RMS alone captures the relevant scaling information. For a 4B model trained on 2T+ tokens, the computational savings of RMSNorm accumulate to a meaningful reduction in total training FLOPs.

Coordinate Special Tokens: Compact Spatial Representation

The coordinate special token system is a non-obvious but impactful engineering optimization that directly addresses a practical bottleneck in the Layout-as-Thought mechanism. The problem it solves: if the model represents bounding box coordinates as plain digit sequences (e.g., "779" for the x-coordinate 0.779 in normalized space), each coordinate requires multiple tokens — "7", "7", "9" — and a single bounding box with four coordinates requires 12+ tokens just for spatial information. For complex documents with 60+ layout elements, this adds hundreds of tokens to the thinking phase, increasing both generation time and the distance between the layout analysis and the final response (making it harder for the model to condition on the layout information).

The solution:

"All bounding box coordinates are normalized to [0, 999] and represented as dedicated special tokens <COORD_0> through <COORD_999>. These tokens are introduced during Stage 3 continued pretraining alongside layout analysis data, enabling the model to learn spatial representations efficiently."

Mechanism: The model's vocabulary is extended with 1,000 new tokens, one for each integer value from 0 to 999. During Stage 3 continued pretraining, these tokens appear in the layout analysis training data, and the model learns their embeddings through standard language model training — the embedding of <COORD_779> encodes the notion of "x-coordinate 0.779" in a continuous vector space, and the model's attention mechanism can learn spatial relationships (e.g., that <COORD_200> is to the left of <COORD_800>) from the training data distribution.

Efficiency impact:

"Compared to encoding coordinates as plain digit sequences (e.g., '779' requires 3 tokens), each coordinate consumes only a single token, reducing the thinking output length by approximately 50% and substantially decreasing inference latency for the layout reasoning phase."

The 50% reduction claim: in the paper's example (Figure 5), a single layout element specification looks like:

<box>[[<COORD_779>,<COORD_057>,<COORD_890>,<COORD_071>]]</box>

without coordinate special tokens, this would be approximately:

<box>[[779,057,890,071]]</box>

The difference is $4 \times (3 \text{ digits} - 1 \text{ special token}) = 8$ tokens saved per element for a 3-digit coordinate. For coordinates with different digit counts (e.g., "57" vs. "890"), the savings vary, yielding the ~50% reduction estimate. For a page with 60 elements, this saves roughly $60 \times 8 \approx 480$ tokens in the thinking phase — directly reducing inference latency proportional to the token count reduction.

Training integration: The coordinate tokens are introduced during Stage 3 (domain-specific continued pretraining) along with the layout analysis data, meaning the model learns coordinate embeddings in the context of document processing tasks. This is important because the semantic meaning of a coordinate — its relationship to other coordinates, to element types, and to the visual features it corresponds to — is task-specific and benefits from being learned alongside the layout analysis objective rather than being randomly initialized and trained only during instruction tuning.

Layout-as-Thought: Recovering Spatial Grounding in End-to-End Models

Layout-as-Thought is the architectural mechanism that addresses the paper's central critique of end-to-end OCR models — the loss of explicit layout analysis — and it operates through a carefully designed two-phase generation process.

Phase 1: Structured Layout Reasoning (triggered by thinking tokens).

When a user prepends thinking to their query, the model enters a layout reasoning mode before producing the final output. The key design decision is that the layout phase is optional — it is not a mandatory preprocessing step but rather a capability that users can invoke for documents where spatial reasoning provides value.

The model generates a structured layout analysis enclosed in <layout>...</layout> tags. The generation is autoregressive: the model produces one layout element at a time, conditioning on previously generated elements, following the reading order of the document. For each element, three fields are generated:

1. Bounding box: <box>[[<COORD_X1>,<COORD_Y1>,<COORD_X2>,<COORD_Y2>]]</box> — four normalized coordinates specifying the element's spatial extent. X1, Y1 are the top-left corner, X2, Y2 are the bottom-right corner, normalized to $[0, 999]$.

2. Element type: <label>element_category</label> — one of the 25 categories from the layout taxonomy, specifying the semantic role of this document region (e.g., paragraph_title, display_formula, table, vision_footnote).

3. Content summary: <brief>concise description</brief> — for text-type elements, a brief natural language summary of the content. This is not verbatim extraction but rather a semantic tag: "Section title indicating review and organization," "Asks to identify shapes seen from the right side," "A 3D arrangement of white cubes with light blue edges." The brevity is intentional — the thinking phase should provide structural guidance, not duplicate the final output.

Reading order is a first-class property:

"The thinking phase enumerates elements following the natural reading order of the document (handling multi-column layouts, interleaved figures, and footnotes), providing an explicit ordering signal that the response generation can follow to produce correctly sequenced output."

This is one of the two mechanisms by which layout reasoning benefits the final output. Reading order determination is non-trivial for complex documents: a two-column academic paper has text that jumps from the bottom of the left column to the top of the right column; a newspaper has interleaved articles with continuation lines; a page with a large figure might have text flowing around it. By explicitly generating reading order during the thinking phase, the model produces a sequentialization of the 2D document that the response generation can simply follow, avoiding the need for the response generation phase to rediscover the reading order from visual features alone.

Phase 2: Response Generation (conditioned on layout reasoning).

Following the </layout> closing tag, the model generates the final response conditioned on both the visual tokens (from the vision encoder) and the self-generated layout analysis (from the thinking phase). This conditioning works through standard autoregressive attention: when generating the response tokens, the model attends to all preceding tokens, including the layout analysis tokens. This means the response generation "sees" the explicit spatial structure that the thinking phase produced and can use it to guide formatting and sequencing.

The paper identifies two mechanisms by which the layout reasoning benefits the final response:

1. Element-type-aware generation:

   "By explicitly identifying element categories (formula, table, image, text), the model applies appropriate rendering formats in the response — wrapping mathematical content in $...$ blocks, converting table structures to HTML, and inserting image placeholders at correct positions."

   This is a formatting guidance mechanism: the model learns during training that when the layout phase labels an element as display_formula, the response should wrap the corresponding mathematical content in block LaTeX delimiters; when it labels an element as table, the response should produce an HTML <table> structure; when it labels an element as image, the response should insert a placeholder and move on.

2. Reading-order-guided sequencing:

   The explicit reading order enumeration ensures that multi-column layouts, interleaved figures, and footnotes appear in the correct sequence in the output. Without this guidance, an end-to-end model might transcribe text in a left-to-right, top-to-bottom raster order — which would intermix columns and misplace footnotes — rather than following the semantic reading order.

When to use Layout-as-Thought (and when not to). The paper's analysis in Figure 4 provides empirical guidance:

"In the high-entropy region (left portion of the curve), where documents contain diverse element types such as mixed text, formulas, tables, and figures, enabling thinking provides a consistent score advantage. As lower-entropy samples are progressively included — documents with more homogeneous layouts (e.g., pure text pages) — the gap narrows and eventually reverses."

The practical recommendation: enable thinking for structurally complex documents (exam papers, technical reports, newspapers with mixed content types), disable it for homogeneous documents (single-column text, simple forms). The "overhead" the paper refers to is not just latency (generating 60+ layout elements before starting the response) but also the risk that explicit layout reasoning interferes with direct recognition — the model might spend capacity reasoning about spatial structure that doesn't help, or might make layout errors that mislead the response generation.

Open questions about Layout-as-Thought. The paper acknowledges (Section 7) that the current implementation has limitations:

"The current implementation generates layout bounding boxes, labels, and brief text descriptions in a relatively rigid format via supervised fine-tuning. Future work should integrate these layout elements more naturally into the reasoning process, allowing the model to flexibly invoke spatial reasoning when needed rather than producing a fixed-format layout dump."

The rigid format (fixed <box>, <label>, <brief> structure) means the model cannot adapt its layout reasoning to the task — it always produces the same kind of analysis regardless of whether the downstream task requires fine-grained spatial localization (e.g., KIE on a specific field) or only coarse structural awareness (e.g., document QA about high-level content). A more flexible system might learn to produce task-adaptive layout reasoning through reinforcement learning, where the layout generation is optimized based on downstream task rewards rather than supervised next-token prediction against a fixed format.

Additionally, the Layout-as-Thought mechanism has only been validated on OmniDocBench v1.5 for document parsing; its effectiveness on other tasks — KIE, document QA, chart understanding — "remains unexplored" (Section 7). The paper hypothesizes that layout reasoning could benefit these tasks, but provides no empirical evidence, leaving this as a key area for future investigation.

Four-Stage Progressive Training Recipe

The training recipe is the paper's most significant methodological contribution: a carefully orchestrated curriculum that builds capabilities sequentially, validated through ablation studies on the 8B model family. Each stage has a distinct purpose, data mixture, training configuration, and set of trainable parameters.

Stage 1: Cross-Modal Alignment.

"Establishes fundamental vision-language alignment with adapter-only training, using basic image-caption pairs and simple OCR tasks to ensure stable initialization."

• Trainable parameters: Only the cross-modal adapter (frozen ViT, frozen Qwen3-4B backbone).

• Token budget: 50B tokens.

• Data composition: Basic image-caption pairs (general vision-language alignment) and simple OCR tasks (single-line text recognition, simple table extraction). The paper does not specify exact ratios, but the emphasis is on establishing the vision-to-language projection before exposing the frozen language model to complex document-specific data.

• Key hyperparameters:

  • Global batch size: 1,024 samples
  • Peak learning rate: $1 \times 10^{-3}$ (the highest in the curriculum, reflecting the need for fast convergence on the randomly initialized adapter)
  • LR schedule: Cosine decay with 0.01 warmup ratio
  • Optimizer: AdamW ($\beta_1 = 0.9$, $\beta_2 = 0.95$), weight decay 0.05
  • Max sequence length: 32,768 tokens

  Why a high learning rate for adapter-only training: The adapter is randomly initialized while the ViT and language model are pretrained with well-conditioned weights. The high learning rate allows the adapter to quickly learn the projection mapping without risking divergence (since the gradients cannot propagate into the frozen components). If the learning rate were too low, the adapter would converge too slowly, requiring more tokens to achieve the same alignment quality.

Stage 2: Foundational OCR Training.

"Develops comprehensive OCR capabilities through full parameter training with OCR-heavy data mixture."

• Trainable parameters: All parameters (ViT, adapter, Qwen3-4B backbone).

• Token budget: 2 trillion tokens — by far the largest stage, accounting for approximately 70% of total training tokens ($2\text{T} / 2.85\text{T} \approx 70\%$).

• Data composition:

  • Document OCR: 45% — structured document parsing into Markdown, covering diverse layouts, languages, and document types
  • Scene OCR: 25% — text in natural images (street signs, product labels, screenshots), providing robustness to non-document visual contexts
  • Caption: 15% — general image captioning, maintaining the vision-language alignment established in Stage 1
  • Specialized OCR: 15% — handwriting recognition, formula recognition, table extraction, multilingual text — covering the specific sub-domains that Stage 3 will later enhance

• Key hyperparameters:

  • Global batch size: 2,048 samples (doubled from Stage 1, exploiting the fact that full-parameter training benefits from larger batches for gradient stability)
  • Peak learning rate: $2 \times 10^{-5}$ (reduced by 50× from Stage 1, reflecting the need to fine-tune pretrained weights without overwriting)
  • LR schedule: Cosine decay with 0.01 warmup ratio
  • Optimizer, weight decay, max sequence length: Same as Stage 1

  Why the data mixture matters: The 45% document OCR + 25% scene OCR + 15% specialized OCR composition means that 85% of tokens are OCR-specific, with only 15% general caption data. This is the stage where the model transitions from a general VLM to an OCR specialist. The caption data serves as a regularizer — without it, the model would rapidly forget general vision-language capabilities, which the ablation study (Table 2) confirms is problematic: pure OCR data in later stages underperforms OCR + General mixtures.

The ablation study validates Stage 2 as essential (Table 2). When Stage 2 is skipped — going directly from Stage 1 to domain-specific Stage 3 training — the average accuracy drops sharply. The paper provides specific numbers: the best Stage 3-only configuration (OCR + General mixture) achieves 80.07%, while Stage 2 followed by Stage 4 alone achieves 83.47%. This 3.4 percentage point gap demonstrates that "large-scale general-purpose pretraining provides a critical capability foundation that cannot be substituted by domain-specific data alone" — a finding that directly motivated the inclusion of Stage 2 in the final recipe.

Stage 3: Domain-Specific Enhancement.

"Implements targeted enhancement for enterprise-critical OCR domains with balanced mixture."

• Trainable parameters: All parameters.

• Token budget: 800B tokens — approximately 28% of total tokens.

• Data composition:

  • Complex Tables: 22%
  • Formula Recognition: 20%
  • Chart Understanding: 18%
  • Information Extraction (KIE): 18%
  • Multilingual OCR: 12%
  • Document Understanding: 10%

  Plus: 70% domain-specific data, 30% general data — maintaining a general data component to prevent catastrophic forgetting, consistent with the ablation finding that general data mixing acts as an "effective regularizer."

• Key hyperparameters:

  • Global batch size: 2,048 samples
  • Peak learning rate: $1 \times 10^{-5}$ (further reduced from Stage 2, reflecting the shift from broad foundation building to targeted specialization — lower learning rate prevents overfitting to narrow domain patterns)
  • Other hyperparameters: Same as Stage 2

Why the specific domain percentages: The distribution reflects enterprise priorities for a document intelligence system. Complex tables and formula recognition are weighted highest because they are the most technically challenging sub-domains and because errors in these areas (merging a table cell incorrectly, misreading a mathematical expression) are particularly costly in downstream applications. Chart understanding and KIE are weighted equally (18% each) because they represent the boundary between pure OCR and document understanding — the capabilities that differentiate Qianfan-OCR from pipeline systems. Multilingual OCR (12%) covers 192 languages, and document understanding (10%) maintains the general comprehension capability.

The 70/30 split is validated by the ablation study (Table 2). In Stage 3, the OCR + General mixture (1:1 ratio) achieves 80.07% while pure OCR data achieves only 75.97% — a 4.1-point gap. When Stage 2 is included, the corresponding gap narrows (84.39% vs. 84.09%) but still favors the mixture. The paper interprets this as: "maintaining general capability during domain specialization acts as an effective regularizer and prevents overfitting to narrow OCR patterns." The 30% general data in the final recipe (rather than 50% in the ablation) represents a compromise between specialization intensity and regularization — the slightly higher domain percentage (70% vs. 50%) reflects the judgment that Stage 2 has already provided sufficient general capability that Stage 3 can tilt more aggressively toward domain specialization.

Stage 4: Instruction Tuning and Reasoning Enhancement.

"Covers a comprehensive set of document intelligence tasks including document parsing, layout analysis, handwriting recognition, scene text recognition, formula recognition, table recognition, multi-page document parsing, chart QA, document QA, and complex table QA."

• Trainable parameters: All parameters.

• Token budget: Not specified in tokens; described as "millions of instruction samples" — substantially smaller than the pretraining stages.

• Data composition: Instruction-tuning samples constructed through three complementary strategies:

  1. Public data curation: Publicly available OCR-related training datasets are collected and instruction-rewritten using DeepSeek models to diversify prompt styles and task formulations. This prevents the model from overfitting to specific prompt patterns (e.g., always expecting "Parse this document:" rather than "Extract the text from this image:" or "Transcribe this page:").
  2. Reverse synthesis: For tasks amenable to reverse generation (tables, exam papers), large-scale QA pairs are constructed by generating questions conditioned on structured ground-truth content. This creates a many-to-one mapping from diverse user queries to consistent outputs, teaching the model to follow instructions rather than simply pattern-match.
  3. Chart data mining: Chart-figure pairs are extracted from academic paper LaTeX sources, and chart understanding QA pairs are generated grounded in the original source code and captions — ensuring that QA pairs are factually anchored rather than hallucinated.

  All instruction data undergoes "systematic prompt generalization and rewriting to improve robustness to diverse user instructions."

• Key hyperparameters:

  • Global batch size: 512 samples (reduced from 2,048, reflecting the smaller dataset size — instruction tuning uses smaller batches to avoid overfitting)
  • Peak learning rate: $1 \times 10^{-5}$ (same as Stage 3)
  • LR schedule: Cosine decay with 0.03 warmup ratio (higher warmup than previous stages — 3% vs. 1% — to ease the transition from continued pretraining to instruction following)
  • Other hyperparameters: Same as Stages 2–3

Training infrastructure and iteration strategy:

"All training is conducted on 1,024 Baidu Kunlun P800 chips using 3D parallelism (data, tensor, and pipeline parallelism with communication-computation overlap), processing over 2.85T tokens across all stages."

The 3D parallelism strategy: data parallelism distributes different batch samples across chips; tensor parallelism splits individual layers across chips (each Transformer layer's attention heads or MLP dimensions are partitioned); pipeline parallelism splits different layers across chips (layers 1–9 on chip group A, layers 10–18 on chip group B, etc.). Communication-computation overlap means that gradient communication between chips happens concurrently with forward/backward computation rather than sequentially — when chip A finishes computing its portion of layer 9, it begins communicating gradients to chip B while simultaneously starting computation on the next micro-batch, hiding communication latency.

"Stages 1 and 2 are trained once to establish a stable foundation checkpoint, while Stages 3 and 4 are iterated multiple times to explore different domain-specific data mixtures, sampling ratios, and instruction tuning configurations."

The iteration strategy is practical: Stages 1+2 account for 2.05T tokens and take most of the training wall-clock time, while Stages 3+4 account for 800B tokens + instruction samples and can be completed much faster. The paper reports that "the full four-stage pipeline completes within a week, and Stage 3/4 iterations take approximately one day each, supporting systematic ablation and optimization of OCR-specific training recipes." This rapid iteration capability is what enabled the paper's ablation studies (Table 2) on the 8B model — multiple Stage 3/4 configurations could be tested in a few days, with the best configuration then validated at the 4B scale.

Key design insight from the ablation study (Table 2). The paper's central training finding is the validity of the progressive curriculum:

• Stage 1 → Stage 4 baseline: 71.37% (adapter alignment + instruction tuning only — catastrophically low, confirming that pretraining is essential)
• Stage 1 → Stage 2 → Stage 4: 83.47% (adding foundational OCR pretraining provides +12.1 points)
• Stage 1 → Stage 2 → Stage 3 → Stage 4: 84.39% (adding domain-specific enhancement provides +0.92 points over the 3-stage curriculum)

The diminishing returns from Stage 3 (+0.92 points) compared to Stage 2 (+12.1 points) reflect the paper's finding that "Stage 2 provides a critical capability foundation that cannot be substituted by domain-specific data alone" — the bulk of OCR capability comes from the large-scale foundational pretraining, with domain-specific enhancement providing incremental refinement.

The paper also notes that this 84.39% exceeds Qwen2.5-VL-7B (79.30%) by +5.09%, despite the comparison model being a strong general-purpose VLM of comparable scale. This validates the claim that OCR-specific training can substantially outperform general VLM training on document tasks at similar model sizes.

The final training recipe for the 4B Qianfan-OCR follows this validated curriculum, adapted from the 8B ablation results. The paper does not specify whether the data mixture ratios were adjusted for the 4B model, but the implication is that the same architecture-family-consistent recipe generalizes across model scales.

What the ablation study does NOT cover. The paper does not ablate:

• The specific percentages within the Stage 2 data mixture (e.g., whether 45% document OCR vs. 35% or 55% would be better)
• The 70/30 domain-to-general split in Stage 3 (only 50/50 and 100/0 were ablated, not intermediate ratios)
• The Stage 1 token budget (50B is used throughout; whether more or fewer tokens would improve alignment is unexplored)
• The effect of the coordinate special token introduction in Stage 3 vs. introducing them earlier or later
• The contribution of individual data synthesis pipelines to overall performance

These represent open questions that the paper's rapid Stage 3/4 iteration framework could address in future work, but the paper does not report systematic experiments exploring these dimensions.

4. Key Insights and Innovations

Innovation 1: Layout-as-Thought Reframes Spatial Grounding as an Optional Reasoning Skill Rather Than an Architectural Prerequisite

The central conceptual move of this paper is decoupling spatial grounding from the detection-first architecture that pipeline systems treat as mandatory for layout analysis. Before Qianfan-OCR, the dominant assumption in the field was that explicit layout analysis — bounding boxes, element type labels, reading order — could only be produced by a dedicated detection module that processes the document before text recognition begins. This assumption was so deeply embedded that it shaped the entire taxonomy of OCR systems: pipeline models (PaddleOCR-VL, MonkeyOCR, MinerU 2.5) perform detection-then-recognition, while end-to-end models (Nougat, DeepSeek-OCR, olmOCR) skip detection entirely and thus lose layout analysis as a capability. The field had implicitly accepted that the choice between these architectures was a choice about whether spatial grounding was available — and that end-to-end models had permanently sacrificed it.

Layout-as-Thought breaks this assumed coupling by treating layout analysis as an optional chain-of-thought reasoning phase rather than a mandatory preprocessing step. The model reasons about spatial structure in natural language (augmented with coordinate special tokens) before generating its final output, exactly as a human might say "first I'll identify what's on this page and where everything is, then I'll transcribe it." This is a fundamental reframing rather than an incremental improvement: it turns layout analysis from an architectural property (something the system is — a detection module) into a behavioral property (something the system does — a reasoning step). The consequence is that spatial grounding becomes optional and context-dependent: users activate it with thinking tokens when the document's layout complexity warrants explicit reasoning, and skip it when the overhead isn't justified.

The significance of this reframing extends beyond the specific mechanism. It suggests a broader principle for end-to-end document intelligence: capabilities that were previously externalized into separate pipeline stages can be internalized as learned reasoning behaviors, controllable through the same prompt interface that governs the rest of the model's behavior. This has implications for how future systems might handle other pipeline-externalized functions — table structure extraction, handwriting style normalization, language identification — as optional reasoning subroutines rather than mandatory architectural components.

The evidence for why this reframing works — and where it fails — comes from the layout label entropy analysis in Figure 4. When the paper sorts OmniDocBench samples by layout complexity (measured as the entropy of element type distribution per page) and plots cumulative score with and without thinking, a clear boundary emerges: thinking provides gains in the high-entropy region (mixed text, formulas, tables, figures — the documents where spatial reasoning resolves genuine ambiguities) but introduces overhead and degrades performance on low-entropy documents (pure text, simple forms — where explicit layout reasoning is unnecessary and potentially distracting). This non-monotonic benefit curve — where the same mechanism helps on hard cases but hurts on easy ones — is precisely the signature of a reasoning tool rather than an architectural necessity. A detection module, by contrast, would process every page identically regardless of complexity. The fact that Layout-as-Thought's utility is difficulty-dependent confirms that it functions as optional reasoning rather than mandatory preprocessing.

This framing also partially reconciles the tension between pipeline and end-to-end advocates. Pipeline proponents argue that explicit spatial grounding is essential for real-world applications. End-to-end proponents argue that unified architectures avoid error propagation and deployment complexity. Layout-as-Thought suggests that these positions are not contradictory: the end-to-end model can perform spatial grounding when it matters, but isn't forced to pay the overhead when it doesn't. The remaining gap — whether the reasoning-based approach can match the raw localization accuracy of dedicated detection modules on arbitrary documents — is an empirical question the paper only partially answers (Layout-as-Thought achieves 92.64 on OmniDocBench vs. 93.12 without, indicating a small aggregate penalty, but the per-document analysis reveals the pattern described above).

Innovation 2: Fine-Grained Layout Taxonomy as a Supervision Signal for Structured Formatting

The paper's choice of a 25-category layout taxonomy — adopted from PaddleOCR-VL's label system over MinerU 2.5's coarser 4-label system — is not a minor hyperparameter choice but rather a deliberate design decision that shapes what the model learns about how to format different document elements. This is a diagnostic contribution: the paper demonstrates empirically that label granularity is not merely aesthetic but directly enables downstream structured formatting capabilities that coarse labels cannot support.

The field has not systematically studied how layout label granularity affects end-to-end OCR model behavior. Prior end-to-end systems (Nougat, olmOCR, DeepSeek-OCR) do not explicitly classify elements during training — they learn to produce structured outputs from document images without intermediate element type labels. Pipeline systems use detection labels internally, but those labels are consumed by rule-based postprocessing stages (e.g., "if label is table, route to table parser; if label is formula, route to LaTeX renderer") rather than by learned neural formatting. Qianfan-OCR's innovation is to use fine-grained labels as a supervision signal for learned formatting behavior, where the model must learn that display_formula elements should be rendered in $$ ... $$ blocks while inline_formula elements should remain in text flow, and that paragraph_title, doc_title, and reference_content warrant different Markdown heading levels or formatting conventions.

The paper provides concrete evidence for why this matters through its comparison of PaddleOCR-VL's and MinerU 2.5's label systems (Section 3.2). The key difference — distinguishing abstract from content, reference from reference_content, aside_text from body text — directly enables structured extraction tasks that coarse labels cannot support. An academic paper parser that cannot distinguish abstracts from body text cannot reliably extract structured metadata. A legal document parser that cannot distinguish section titles from body text cannot produce a hierarchical outline. These are not edge cases; they are core functionality for document intelligence applications.

The significance of this insight is methodological rather than architectural: it establishes that the choice of layout taxonomy in OCR training data has downstream consequences for model capabilities that are not obvious from recognition accuracy metrics alone. A model trained with 4 coarse labels might achieve the same OmniDocBench score as one trained with 25 fine-grained labels — the benchmark's edit-distance metrics might not penalize coarse formatting — but would fail on structured extraction tasks that require element type discrimination. This implies that benchmark design for OCR systems should incorporate structured extraction subtasks that are sensitive to label granularity, rather than relying solely on text-edit and structure-preservation metrics.

The choice also reflects a pragmatic engineering principle that the paper articulates implicitly but doesn't state as a general claim: when using teacher models to generate training data, prefer the teacher with finer-grained output labels, even at the cost of reduced detection accuracy in some edge cases, because finer labels enable the student to learn richer formatting behaviors. The paper notes that PaddleOCR-VL achieves "consistently higher detection accuracy" on the evaluation benchmark, but the more important property is the label system's semantic granularity — a property that detection accuracy benchmarks may not capture.

Innovation 3: The Inferiority of Two-Stage Pipelines on Visual-Spatial Reasoning Tasks Is an Architectural Property, Not an Accuracy Problem

The paper provides the most comprehensive empirical evidence to date that the performance degradation of two-stage OCR+LLM pipelines on document understanding tasks is not a correctable accuracy issue but a fundamental consequence of discarding visual context during text extraction. This is not a new claim — prior work has argued that visual context matters for document understanding — but the paper's systematic evaluation across eight benchmarks (Table 6) quantifies the damage with unusual clarity and specificity, establishing a diagnostic that the field can use to evaluate architectural claims.

The key pattern in Table 6: two-stage systems built by pairing specialized OCR models (PaddleOCR-VL, MinerU 2.5, Dotsocr, DeepSeek-OCR) with Qwen3-4B as the downstream LLM achieve zero accuracy on CharXiv (both descriptive and reasoning questions), dramatically lower scores on ChartQA (7.0–56.8 vs. 88.1 for Qianfan-OCR), and significantly lower scores on ChartBench (2.0–17.2 vs. 85.9). These are not marginal degradations — they represent catastrophic failure on tasks where visual-spatial information (chart structure, axis relationships, data point positions, spatial layout) is essential for answering questions. The degradation persists even on text-heavy tasks like DocVQA (55.9–67.1 vs. 92.8), confirming that spatial and layout context provides value beyond what plain text can capture.

What makes this finding significant is its implication for the architecture debate. Pipeline advocates could argue that the accuracy gap on OmniDocBench (94.50 for PaddleOCR-VL 1.5 vs. 93.12 for Qianfan-OCR) represents a meaningful advantage for detection-first approaches. The counterargument from Table 6 is that this accuracy advantage comes at the cost of rendering the system incapable of performing a broad class of document understanding tasks that require visual reasoning. A 1.38-point advantage on structured parsing is arguably less important than a 60+ point disadvantage on chart understanding, if the deployment scenario requires both capabilities. The paper's contribution is to make this trade-off explicit and quantifiable, converting what was previously a qualitative debate ("end-to-end models might lose some accuracy but gain understanding") into an empirical comparison with specific numbers.

The finding also provides a boundary condition for when pipeline architectures are sufficient: if the deployment task is pure text extraction from well-structured documents with no downstream understanding requirement, pipeline systems remain highly competitive and may be preferable due to lower model complexity. But as soon as the task requires any form of visual-spatial reasoning — even tasks that appear text-heavy on the surface, like DocVQA — the pipeline architecture's irreversible loss of visual context becomes a hard ceiling that no amount of recognition accuracy improvement can overcome. This boundary condition is not obvious a priori; the paper's contribution is to demonstrate it empirically.

Innovation 4: Multi-Stage Progressive Training Validates That OCR-Specialized Pretraining Is a Distinct Capability Phase Separate from Instruction Tuning

The paper's ablation study (Table 2) provides controlled evidence for a phase transition in capability acquisition during training: foundational OCR pretraining (Stage 2) provides the bulk of OCR capability (+12.1 points over the Stage 1 → Stage 4 baseline), while domain-specific enhancement (Stage 3) provides incremental refinement (+0.92 points), and instruction tuning (Stage 4) provides task-following behavior but not fundamental OCR capability. This finding has methodological significance beyond the specific model: it establishes that OCR capability in end-to-end VLMs is acquired primarily during large-scale pretraining on OCR-specific data, not during instruction tuning, and that skipping the pretraining phase cannot be compensated for by more domain-specific data or better instruction tuning.

The field has an ongoing debate about whether specialized capabilities in VLMs are best acquired through continued pretraining (massive token budgets on domain-specific data) or through instruction tuning (smaller datasets of high-quality task demonstrations). The STaR/ReST$^{EM}$ line of work (Zelikman et al., 2022; Singh et al., 2024) argues that iterative self-improvement through instruction-style fine-tuning can bootstrap complex reasoning capabilities. The olmOCR line of work uses large-scale SFT on web documents. The paper's ablation provides a concrete data point: when Stage 2 is skipped, even the best Stage 3 configuration (OCR + General mixture at 80.07%) falls substantially below Stage 2 → Stage 4 (83.47%). The gap cannot be closed by better domain-specific data mixing — it requires the massive token budget and OCR-heavy composition of Stage 2.

The practical implication is that building an effective end-to-end OCR model requires a compute investment comparable to pretraining a general VLM, not merely fine-tuning one. The 2T tokens of Stage 2 represent ~70% of the total training budget, dwarfing the instruction tuning phase in both compute and data volume. Organizations hoping to adapt general VLMs for OCR through instruction tuning alone — a common approach given the ease of constructing instruction datasets compared to pretraining corpora — will hit a capability ceiling that no amount of instruction data quality can overcome, because the model never acquires the fundamental visual-textual alignment for dense document processing that pretraining provides.

The finding also suggests an architecture-agnostic principle: capabilities that require learning new perceptual mappings (pixels to structured text, visual layout to reading order, chart images to data semantics) require pretraining-scale data budgets, while capabilities that only require learning new task formats (following instructions, producing output in specific formats) can be acquired through instruction tuning. This is a refinement of the standard transfer learning paradigm: fine-tuning adapts the model's behavior, but pretraining establishes its perceptual competence. The paper's contribution is to provide quantitative evidence for where this boundary lies in the specific context of document intelligence, using a controlled ablation that isolates the contribution of each training phase.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary recognition benchmark is OmniDocBench v1.5 (Ouyang et al., 2024), a diverse PDF document parsing benchmark with comprehensive per-element annotations (text, formulas, tables, reading order). The paper also evaluates on OlmOCR Bench (AllenAI, 2024) for end-to-end document OCR across categories including arXiv papers, old scans, tables, multi-column layouts, and base documents. General OCR capability is measured on OCRBench (Chen et al., 2024a), OCRBench v2 (English and Chinese), and CCOCR (multilingual and overall). Document understanding is assessed on eight benchmarks: OCRVQA, TextVQA_VAL, DocVQA, CharXiv (descriptive and reasoning questions), ChartQA_TEST, ChartQAPro, and ChartBench. Key Information Extraction (KIE) covers five public benchmarks: OCRBench KIE, OCRBenchv2 KIE (English), OCRBenchv2 KIE (Chinese), CCOCR KIE, and Nanonets KIE (F1). The specific test splits and sample counts are not enumerated for all benchmarks; OmniDocBench v1.5 is the most thoroughly analyzed with per-metric breakdowns, per-sample layout entropy analysis, and cross-model comparisons. The paper does not explicitly state the number of test samples in most benchmarks, relying instead on benchmark-standard splits.

• Base model(s). The primary model is Qianfan-OCR, a 4B-parameter vision-language model built on the Qianfan-VL multimodal bridging architecture (Dong et al., 2025) with a Qianfan-ViT vision encoder, a two-layer MLP cross-modal adapter, and Qwen3-4B (Bai et al., 2025) as the language model backbone. The vision encoder uses AnyResolution dynamic tiling into 448×448 patches with up to 16 tiles (4,096 visual tokens maximum). For the ablation study (Table 2), the paper uses Qianfan-VL-8B — a model from the same architectural family that has undergone large-scale general-purpose continual pretraining — as a lower-cost proxy to validate the training recipe before scaling to the 4B model. The 8B model shares the same architectural design (ViT + adapter + language model) but with a larger backbone, making it a suitable ablation platform since findings about training stage contributions are argued to generalize across model scales within the family.

• Metrics. For OmniDocBench v1.5, the paper reports six metrics following the official evaluation protocol: Overall score (higher is better), TextEdit (normalized edit distance, lower is better), FormulaCDM (formula recognition accuracy via CDM metric, higher is better), TableTEDs (table structure recognition via Tree Edit Distance, higher is better), TableTEDss (table structure with content, higher is better), and R-orderEdit (reading order edit distance, lower is better). The Overall score is a composite metric; the paper does not disclose the exact weighting but follows the official OmniDocBench evaluation. For OlmOCR Bench, per-category accuracy scores are reported (ArXiv, Old scans, Tables, Old math scans, Headers & footers, Multi column, Long tiny text, Base) along with an unweighted average across categories. For general OCR benchmarks (OCRBench, OCRBenchv2, CCOCR), accuracy-based metrics are computed using VLMEvalKit (Duan et al., 2024) following benchmark-specific protocols. For document understanding benchmarks, standard accuracy/F1 metrics are used per benchmark convention. For KIE, all scores are normalized to a 0–100 scale with the Overall mean computed as the unweighted average across five benchmarks.

• Baselines. The paper compares against three architectural categories, with models listed explicitly in Tables 3–8:

  • Pipeline OCR Systems: PaddleOCR-VL (Cui et al., 2025b), PaddleOCR-VL 1.5 (Cui et al., 2026), MonkeyOCR (multiple variants: 1.2B, 3B, pro-1.2B, pro-3B), Dolphin, Dolphin-1.5, PP-StructureV3, MinerU2.5, MinerU2-pp, Marker-1.8.2. For OlmOCR Bench (Table 3), PaddleOCR-VL and PaddleOCR-VL-1.5 and MonkeyOCR serve as pipeline baselines.
  • End-to-end Models: DeepSeek-OCR (Wei et al., 2025), DeepSeek-OCR-v2 (Wei et al., 2026), olmOCR (Poznanski et al., 2025), Dotsocr, OCRVerse, OCRFlux, POINTS-Reader, InternVL3 (Zhu et al., 2025), InternVL3.5-241B, MinerU2-VLM, Nanonets-OCR-S, GPT-4o, Gemini-2.5 Pro, Gemini-3 Pro, Qwen2.5-VL-72B (Bai et al., 2025), Qwen3-VL-2B, Qwen3-VL-4B, Qwen3-VL-235B.
  • Commercial and Large Open-Source Models (for KIE): Gemini-3.1-Pro, Gemini-3-Pro, Seed-2.0, Qwen3-4B-VL, Qwen3-VL-235B-A22B.

  For the two-stage OCR+LLM comparison on document understanding (Table 6), the paper constructs baseline systems by pairing four specialized OCR models (PaddleOCR-VL, MinerU2.5, Dotsocr, DeepSeek-OCR) with Qwen3-4B as the downstream LLM. This controlled comparison isolates the architectural effect (end-to-end vs. pipeline) while holding the language model scale constant.

• Generation budget / compute accounting. The paper does not standardize comparisons by generation budget or FLOP count across models, since the compared systems span heterogeneous architectures (pipeline systems with separate detection/recognition modules, end-to-end VLMs of varying sizes, commercial models with undisclosed compute). Instead, comparisons are made on output quality metrics (accuracy, edit distance, F1) under each model's standard inference configuration. The only compute-aware comparison appears in Table 8 (Inference Throughput), where pages per second (PPS) is measured on OmniDocBench v1.5 using a single NVIDIA A100 GPU with vLLM 0.10.2, comparing Qianfan-OCR (W16A16 and W8A8 quantized) against pipeline systems (PaddleOCR-VL, MinerU 2.5, MonkeyOCR-pro-1.2B, Dots OCR) with throughput numbers sourced from the PaddleOCR-VL technical report. No FLOPs-matched or compute-budget-controlled accuracy comparison is conducted — the throughput comparison is purely for deployment feasibility assessment, not for normalized capability evaluation.

• Cross-validation / statistical protocol. No cross-validation is reported for the main evaluation results. The comparisons in Tables 3–7 present single-run scores on benchmark test sets. The only cross-validation-like procedure appears in the training ablation study (Table 2), where different Stage 3 data mixture configurations are compared on "multiple OCR benchmarks" with average accuracy computed — but the paper does not specify whether these are validation-set or test-set numbers, the number of runs, or any variance estimates. The Layout-as-Thought analysis (Figure 4) sorts OmniDocBench v1.5 test samples by layout label entropy and computes cumulative scores — this is a post-hoc analysis on test-set results, not a cross-validated protocol. The paper provides no confidence intervals, standard deviations, or statistical significance tests for any reported result, making it impossible to assess whether differences between models (e.g., Qianfan-OCR at 93.12 vs. PaddleOCR-VL 1.5 at 94.50 on OmniDocBench) are statistically reliable or within benchmark noise.

Main Quantitative Results

OCR-Specific Benchmarks: OmniDocBench v1.5 and OlmOCR Bench

The headline result: Qianfan-OCR achieves 93.12 overall on OmniDocBench v1.5, ranking first among all end-to-end models and second overall behind only the pipeline system PaddleOCR-VL 1.5 (94.50). This represents a 1.38-point gap with the top pipeline system — substantially narrower than the previous best end-to-end result (DeepSeek-OCR-v2 at 91.09, a 3.41-point gap). The per-metric breakdown in Table 4 reveals where the gains come from:

• TextEdit (lower is better): Qianfan-OCR achieves 0.041, matching PaddleOCR-VL 1.5's 0.035 and substantially better than DeepSeek-OCR-v2 (0.048), dots.ocr (0.048), and Gemini-3 Pro (0.065). TextEdit is a normalized character-level edit distance; 0.041 means ~4.1% of characters require edit operations to match ground truth. The strong TextEdit score indicates high raw transcription accuracy.

• FormulaCDM (higher is better): 92.43, second to PaddleOCR-VL 1.5 (94.21) and ahead of DeepSeek-OCR-v2 (90.31). The ~1.8-point gap with the top pipeline system on formulas is one of the largest remaining disparities, suggesting mathematical notation remains a relative weakness for the end-to-end approach.

• TableTEDs (higher is better): 91.02, competitive with top pipeline systems (PaddleOCR-VL 1.5: 92.76, PaddleOCR-VL: 90.89) and substantially ahead of other end-to-end models (DeepSeek-OCR-v2: 87.75, dots.ocr: 86.78). Table structure recognition — measured by Tree Edit Distance between predicted and ground-truth HTML table trees — is a critical metric for real-world document processing, and Qianfan-OCR's strong performance here is notable given that end-to-end models have no dedicated table parsing module.

• TableTEDss (higher is better): 93.85, again second to PaddleOCR-VL 1.5 (95.79) and competitive with top pipeline systems. This metric extends TEDs to include cell content accuracy, making Qianfan-OCR's score evidence that the model preserves both table structure and cell-level text.

• R-orderEdit (lower is better): 0.049, slightly behind PaddleOCR-VL 1.5 (0.042) and PaddleOCR-VL (0.043), but ahead of DeepSeek-OCR-v2 (0.057) and most end-to-end models. Reading order accuracy — whether elements are transcribed in the correct sequence — is a known weakness of end-to-end models that lack explicit layout analysis. Qianfan-OCR's R-orderEdit improvement over other end-to-end models is plausibly attributable to Layout-as-Thought's reading-order-guided sequencing during the thinking phase.

Comparison with specific models in Table 4:

• Vs. DeepSeek-OCR-v2 (91.09): Qianfan-OCR leads by 2.03 points overall, with advantages across every sub-metric except the narrowly comparable TextEdit (0.041 vs. 0.048). The largest gaps are in TableTEDs (91.02 vs. 87.75, +3.27) and TableTEDss (93.85 vs. 92.06, +1.79), suggesting that Qianfan-OCR's complex table synthesis pipeline and Layout-as-Thought mechanism provide genuine gains on structured table extraction.

• Vs. Gemini-3 Pro (90.33): Qianfan-OCR leads by 2.79 points, with particularly large advantages in TextEdit (0.041 vs. 0.065, a ~37% relative reduction in edit distance), TableTEDs (91.02 vs. 88.28, +2.74), and TableTEDss (93.85 vs. 90.29, +3.56). The TextEdit gap is noteworthy because Gemini-3 Pro is a large general-purpose VLM — its weaker raw transcription accuracy on dense documents confirms the paper's claim that general VLMs "underperform specialized systems on structure-sensitive metrics."

• Vs. Qwen3-VL-4B (86.78): Qianfan-OCR leads by 6.34 points, highlighting the effect of OCR-specialized training. Qwen3-VL-4B shares the same language model backbone (Qwen3-4B) but lacks OCR-specific pretraining and Layout-as-Thought. The per-metric gaps are substantial across the board: TextEdit (0.041 vs. 0.055), TableTEDs (91.02 vs. 79.29, +11.73), TableTEDss (93.85 vs. 84.26, +9.59), R-orderEdit (0.049 vs. 0.084). The enormous TableTEDs gap (+11.73) is the single largest per-metric advantage and strongly supports the paper's claim that general VLM training does not suffice for structured document parsing — the table structure recognition deficit of general VLMs is not marginal but catastrophic.

• Vs. pipeline leader PaddleOCR-VL 1.5 (94.50): Qianfan-OCR trails by 1.38 points overall, with the gap concentrated in FormulaCDM (92.43 vs. 94.21, −1.78) and R-orderEdit (0.049 vs. 0.042). The TextEdit and table metrics are surprisingly close (TextEdit: 0.041 vs. 0.035; TableTEDs: 91.02 vs. 92.76). This suggests that the remaining gap between end-to-end and pipeline architectures is primarily in domain-specific areas (math formulas) and explicit reading-order determination, not in general text transcription or table structure recovery — a much narrower and more targeted deficit than the field might have expected.

OlmOCR Bench results (Table 3): Qianfan-OCR achieves 79.8 overall, ranking first among end-to-end models and competitive with the top pipeline system PaddleOCR-VL (80.0). Notable per-category results:

• Base: 99.6, highest across all models (PaddleOCR-VL: 98.5, DeepSeek-OCR: 99.8). Near-ceiling performance on clean, well-structured base documents.
• Old scans: 42.0, best among all end-to-end models (previous best: dots.ocr at 40.9) and competitive with pipeline systems (PaddleOCR-VL: 37.8). The improvement on degraded historical documents — which have noise, faded text, and non-standard layouts — is attributable to the document image augmentation pipeline's noise modeling (broken strokes, ink bleeding, background texture).
• Headers & footers: 92.2, substantially ahead of most end-to-end models (DeepSeek-OCR: 96.1, PaddleOCR-VL: 97.0). While trailing pipeline systems here, the score is still high in absolute terms.
• Multi column: 80.4, competitive with end-to-end models (dots.ocr: 82.4) and pipeline systems (PaddleOCR-VL: 79.9).
• Tables: 81.6, competitive with end-to-end models (POINTS-Reader: 86.0) and ahead of some pipeline systems (MonkeyOCR: 74.6).
• Old math scans: 42.0, best among end-to-end (previous: Qwen3-VL-4B at 40.7, dots.ocr at 40.9).

Layout-as-Thought analysis (Figure 4): This is the paper's most detailed diagnostic result. When OmniDocBench v1.5 samples are sorted by layout label entropy (descending) and cumulative score is plotted:

• High-entropy region (left portion): The "think" variant maintains a consistent score advantage over the "no-think" variant. For the first ~100–200 samples (the most structurally complex documents), the think mode achieves higher cumulative scores. This validates the paper's claim that Layout-as-Thought provides "targeted benefits on structurally complex documents."

• As lower-entropy samples are progressively included: The gap narrows and eventually reverses. The no-think mode achieves a higher total cumulative score (~93.12 vs. ~92.64) once all 500+ samples are included. The paper attributes this reversal to Layout-as-Thought introducing "unnecessary overhead on structurally simple documents, where explicit layout reasoning provides no additional benefit and may even interfere with direct recognition."

• Quantitative detail from the paper: The think variant shows improvements on table-related metrics specifically (TableTEDs: 91.21 → 91.02? Actually, the paper reports "TableTEDs: 91.21 vs. 91.02, TableTEDss: 94.03 vs. 93.85" — but this shows the think variant underperforming on tables, which contradicts the text claim of improvement). On re-reading: "Compared to the default mode, the thinking variant achieves an overall score of 92.64 vs. 93.12, with per-metric results: TextEdit 0.052 vs. 0.041, FormulaCDM 91.92 vs. 92.43, TableTEDs 91.21 vs. 91.02, TableTEDss 94.03 vs. 93.85, and R-orderEdit 0.051 vs. 0.049." The paper then states "the thinking variant shows improvements on table-related metrics (TableTEDs: +0.19, TableTEDss: +0.18)" — but this appears to be an error in the paper. The numbers show TableTEDs 91.21 (think) vs. 91.02 (no-think) = −0.19, and TableTEDss 94.03 (think) vs. 93.85 (no-think) = +0.18. So TableTEDss improves but TableTEDs degrades in think mode. The per-sample entropy analysis presumably disaggregates these effects, showing improvements on high-entropy table-containing documents and degradation on simpler table-free pages, but this disaggregation is only presented visually in Figure 4 for the overall score, not for per-metric breakdowns.

The paper's Layout-as-Thought analysis is thorough for OmniDocBench but, as the paper itself acknowledges, the mechanism's effectiveness on other tasks — KIE, document QA, chart understanding — "remains unexplored" (Section 7). This is a significant gap: Layout-as-Thought is the paper's primary architectural innovation, and its validation is limited to a single benchmark with no evidence that the layout reasoning transfers to downstream tasks that would plausibly benefit from spatial grounding.

General OCR Benchmarks (Table 5)

Qianfan-OCR achieves 880 on OCRBench, ranking first among all compared models (Qwen3-VL-4B: 873, MonkeyOCR: 655, Dotsocr: 625, POINTS-Reader: 620). On OCRBenchv2, the results are mixed:

• English: Qwen3-VL-4B leads (60.68 vs. 56.0 for Qianfan-OCR), a 4.68-point gap favoring the general VLM.
• Chinese: Qianfan-OCR leads (60.77 vs. 59.13 for Qwen3-VL-4B), a narrower 1.64-point advantage.

The paper characterizes this as "Qianfan-OCR prioritizes specialized OCR capabilities while retaining competitive general performance without significant degradation" — a framing that acknowledges the trade-off. On CCOCR, Qianfan-OCR leads both sub-metrics: CCOCR-multilan (76.7 vs. 74.2 for Qwen3-VL-4B) and CCOCR-overall (79.3 vs. 76.5 for Qwen3-VL-4B).

The specialized OCR model degradation on general benchmarks: All specialized OCR pipeline and end-to-end models (PaddleOCR-VL, MinerU2.5, Dotsocr, MonkeyOCR, POINTS-Reader, DeepSeek-OCR) show dramatically lower scores than general VLMs on OCRBenchv2 and CCOCR, despite their strong performance on domain-specific benchmarks like OmniDocBench. PaddleOCR-VL scores only 18.15/40.86 on OCRBenchv2 en/zh (vs. Qianfan-OCR's 56.0/60.77) and 29.1 on CCOCR-overall (vs. 79.3). This confirms that these benchmarks "include not only pure OCR recognition but also understanding and key information extraction (KIE) tasks, where specialized OCR models generally underperform." The paper's methodology note explains that for these benchmarks, "since specialized OCR models typically do not report these metrics, we integrate their OCR outputs in our evaluation environment" — meaning the specialized models' text extraction outputs are fed into the evaluation pipeline rather than being evaluated as end-to-end systems. The extremely low scores suggest these models cannot perform the understanding subtasks that general OCR benchmarks require.

Document Understanding Benchmarks (Table 6)

The most striking result in the paper: two-stage OCR+LLM systems achieve zero accuracy on CharXiv (both descriptive and reasoning questions). This is not a marginal degradation but a complete failure mode. The pattern across all eight benchmarks:

End-to-end Qianfan-OCR performance:

• OCRVQA_TESTCORE: 66.8 (best)
• TextVQA_VAL: 80.0 (second to Qwen3-VL-4B at 81.8)
• DocVQA: 92.8 (second to Qwen3-VL-4B at 94.9)
• CharXiv_DQ: 94.0 (best, by 12.2 points over Qwen3-VL-4B at 81.8)
• CharXiv_RQ: 85.2 (best, by 36.7 points over Qwen3-VL-4B at 48.5)
• ChartQA_TEST: 88.1 (best, by 4.8 points over Qwen3-VL-4B at 83.3)
• ChartQAPro: 42.9 (best, by 6.7 points over Qwen3-VL-4B at 36.2)
• ChartBench: 85.9 (best, by 11.0 points over Qwen3-VL-4B at 74.9)

Qianfan-OCR achieves best scores on six of eight benchmarks, with particular dominance on chart and academic reasoning tasks. The CharXiv results are remarkable: 94.0 on descriptive questions and 85.2 on reasoning questions, dramatically outperforming Qwen3-VL-4B (81.8 and 48.5 respectively). Since CharXiv questions require understanding chart structures, axis relationships, data point positions, and visual encodings, this dominance is directly attributable to the end-to-end architecture's preservation of visual context — information that two-stage systems discard and that general VLMs may not fully exploit without OCR-specific training on chart understanding.

Two-stage OCR+LLM systems:

• On CharXiv: 0.0 across all four OCR backends (PaddleOCR-VL, MinerU2.5, Dotsocr, DeepSeek-OCR) paired with Qwen3-4B. The paper explains: "chart structures, axis relationships, and data point positions – discarded during text extraction – are essential for answering."
• On ChartQA: Scores range from 7.0 (MinerU2.5+Qwen3-4B) to 56.8 (PaddleOCR-VL+Qwen3-4B). The best pipeline combination reaches only 64.4% of Qianfan-OCR's 88.1.
• On ChartBench: Scores range from 2.0 (Dotsocr+Qwen3-4B) to 17.2 (PaddleOCR-VL+Qwen3-4B) — a near-total failure. Qianfan-OCR achieves 85.9.
• On DocVQA: Even this text-heavy task shows substantial degradation: 55.9–67.1 for pipeline systems vs. 92.8–94.9 for end-to-end models. The ~25-point gap suggests that spatial layout information — paragraph ordering, section boundaries, figure-caption associations — provides value even when the questions are ostensibly about document text.

The CharXiv zero-accuracy result is the paper's strongest evidence for the architectural thesis. It demonstrates not a marginal improvement from preserving visual context, but a binary capability threshold: two-stage systems cannot answer chart questions at all, while end-to-end systems can. This converts the qualitative debate ("end-to-end might be better for understanding") into an empirical necessity: if chart understanding is a requirement, pipeline architectures are non-viable.

Qwen3-VL-4B as a comparison point: The general VLM of the same scale outperforms Qianfan-OCR on two benchmarks: DocVQA (94.9 vs. 92.8, −2.1) and TextVQA (81.8 vs. 80.0, −1.8). These are text-heavy document understanding tasks where spatial reasoning is less critical, suggesting that general VLM training provides a slight advantage for pure text comprehension. Qianfan-OCR's OCR-specialized training creates a trade-off: improved performance on structure-sensitive tasks (charts, academic figures, tables) at the cost of minor regression on text-heavy VQA. The paper explicitly acknowledges this: "as a model designed for specialized OCR tasks, Qianfan-OCR accepts modest performance differences on general understanding benchmarks relative to same-size general VLMs."

Key Information Extraction Benchmarks (Table 7)

Qianfan-OCR achieves an overall mean score of 87.9 across five KIE benchmarks, the highest among all compared models:

• Vs. Qwen3-VL-4B (83.5): +4.4 points. The same-scale general VLM comparison is particularly informative: Qianfan-OCR's advantage is driven by Chinese KIE (82.3 vs. 71.3, +11.0 on OCRBenchv2 KIE zh) and Nanonets KIE F1 (86.5 vs. 83.3, +3.2), partially offset by a small gap on OCRBenchv2 KIE English (82.8 vs. 82.1, +0.7).
• Vs. Qwen3-VL-235B-A22B (84.2): +3.7 points. The much larger model (50×+ activated parameters) achieves lower overall performance primarily due to weak Chinese KIE (62.9 on OCRBenchv2 KIE zh vs. Qianfan-OCR's 82.3, −19.4). The larger model does lead on OCRBenchv2 KIE English (85.6 vs. 82.8), suggesting scale provides benefits for English extraction but not Chinese — a finding the paper attributes to "limited multilingual generalization" in the larger model.
• Vs. Gemini-3.1-Pro (79.2): +8.7 points. The commercial model achieves the highest OCRBench KIE score (96.0 vs. Qianfan-OCR's 95.0) but drops sharply on Chinese KIE (63.4 vs. 82.3) and Nanonets F1 (76.1 vs. 86.5). This cross-lingual asymmetry is a recurring pattern: English-optimized models (Gemini series, Seed-2.0) cluster in the 87–96 range on English KIE but collapse to 48–63 on Chinese. Qianfan-OCR achieves 82.3 on Chinese KIE, demonstrating that the multilingual OCR data synthesis pipeline (covering 192 languages with writing-system-specific handling) provides genuine cross-lingual robustness.

A notable methodological point: the KIE benchmarks include both pure extraction tasks (extracting structured fields from documents) and understanding tasks (answering queries about field values). The paper's strong performance across both — coupled with the specialized OCR models' absence from this comparison (since they "lack native KIE capabilities") — supports the claim that end-to-end unification enables capabilities that pipeline architectures cannot provide without additional LLM components.

Inference Throughput (Table 8)

The throughput comparison is not a primary accuracy result but addresses the deployment feasibility question. Key numbers:

• Qianfan-OCR (W8A8 quantization): 1.024 pages per second (PPS) on OmniDocBench v1.5 with a single A100 GPU, batch size 512.
• PaddleOCR-VL (pipeline): 1.224 PPS (from PaddleOCR-VL technical report).
• MinerU 2.5 (pipeline): 1.057 PPS.
• MonkeyOCR-pro-1.2B (pipeline): 0.673 PPS.
• Dots OCR: 0.352 PPS.
• Qianfan-OCR (W16A16, no quantization): 0.503 PPS.

The W8A8 quantized Qianfan-OCR achieves throughput comparable to the fastest pipeline system (1.024 vs. 1.224 PPS, a ~16% gap) while substantially exceeding other pipeline systems. The W8A8 variant provides a 2× speedup over the W16A16 baseline (1.024 vs. 0.503) with what the paper describes as "negligible accuracy degradation" — though no accuracy numbers are reported for the quantized model, so the exact accuracy-cost trade-off cannot be assessed from the paper's data.

The paper attributes this competitive throughput to three architectural advantages: (1) GPU-centric computation eliminates the CPU-based layout analysis bottleneck that throttles pipeline systems under high concurrency; (2) efficient batching of whole-page images with uniform dimensions enables large-batch inference with well-aligned memory access patterns, whereas pipeline systems process variable numbers of cropped regions per page leading to irregular batch sizes and fragmented GPU utilization; (3) lower deployment complexity — a single vLLM instance vs. asynchronous orchestration of heterogeneous stages. These claims are plausible but not empirically verified through controlled ablation (e.g., measuring pipeline system throughput at varying batch sizes, quantifying the CPU bottleneck's impact, or comparing end-to-end latency at different concurrencies).

The throughput numbers should be interpreted with care: PaddleOCR-VL's 1.224 PPS is achieved with a much smaller model footprint (0.9B parameters including detection and recognition modules) compared to Qianfan-OCR's 4B language model backbone. That the end-to-end approach approaches parity despite the 4.4× parameter count difference is notable, but the throughput comparison does not control for total model size or GPU memory consumption — a 4B model requires substantially more VRAM than a 0.9B pipeline system, which affects feasible batch sizes and multi-model serving on a single GPU.

Ablation Studies and Robustness Checks

Multi-stage training effectiveness (Table 2): All configurations evaluated on Qianfan-VL-8B (not the final 4B model). The baseline (Stage 1 adapter alignment + Stage 4 instruction tuning, no foundational pretraining) achieves 71.37% average accuracy. Adding Stage 2 foundational pretraining (general data) followed by Stage 4 raises this to 83.47% (+12.10 points). The complete four-stage pipeline (Stage 1 → Stage 2 General → Stage 3 OCR+General 1:1 mixture → Stage 4) achieves 84.39% (+13.02 points over baseline). The diminishing returns from Stage 3 (+0.92 over Stage 2→Stage 4) compared to Stage 2 (+12.10 over Stage 1→Stage 4) are quantified, establishing that foundational OCR pretraining is the dominant capability driver.

OCR-specific vs. general data mixing in Stage 3 (Table 2): Pure OCR data in Stage 3 achieves lower average accuracy than mixtures including general data. When Stage 2 is not present, Stage 3 OCR+General (80.07%) outperforms Stage 3 OCR-only (75.97%) by 4.10 points. When Stage 2 is present, the gap narrows (84.39% vs. 84.09%) but the mixture still benefits. The paper interprets this as evidence that general data acts as an "effective regularizer" preventing overfitting to narrow OCR patterns. No intermediate mixture ratios (e.g., 70/30, 80/20) are reported — only 0/100, 50/50, and 100/0.

Stage 2 data composition (Table 2): The Stage 2 General configuration (83.47%) outperforms Stage 2 OCR+General mixture (80.07% when followed by Stage 4) suggesting that Stage 2 benefits more from broad general-purpose data than from OCR-specific data, with domain specialization best deferred to Stage 3. This finding directly motivated the training recipe for the 4B model: Stage 2 uses general data to build broad capability, then Stage 3 applies OCR-specific specialization with a 70/30 domain-to-general split.

Layout-as-Thought per-metric analysis (Section 6.1, per-metric breakdown): The thinking variant achieves an overall OmniDocBench score of 92.64 vs. 93.12 for no-think. Per-metric: TextEdit degrades (0.052 vs. 0.041), FormulaCDM degrades (91.92 vs. 92.43), TableTEDs degrades (91.21 vs. 91.02), TableTEDss improves (94.03 vs. 93.85, +0.18), R-orderEdit degrades (0.051 vs. 0.049). The paper's claim that "the thinking variant shows improvements on table-related metrics" is misleading — only TableTEDss improves, and by a small margin. The layout entropy analysis (Figure 4) provides the crucial nuance: think mode helps on structurally complex documents but hurts on simple ones, and the overall negative aggregate score reflects the fact that OmniDocBench contains more simple documents than complex ones.

Layout label entropy analysis (Figure 4): This is the paper's most informative robustness check. By sorting samples by layout complexity and plotting cumulative scores, the analysis reveals that Layout-as-Thought's benefit is conditional on document heterogeneity. The paper does not provide a precise entropy threshold where think mode becomes beneficial — only the qualitative pattern from the cumulative curve. A more actionable analysis would specify the entropy value at which the think mode's accuracy crosses above or below the no-think mode, enabling users to make data-driven decisions about when to activate it.

Comparison of Qwen3-VL-4B (same backbone, no OCR specialization) across all benchmarks: This functions as an implicit ablation of OCR-specific training. Qwen3-VL-4B shares the Qwen3-4B language model backbone but lacks the OCR-specific pretraining (Stage 2), domain-specific enhancement (Stage 3), and Layout-as-Thought. The performance gap between Qianfan-OCR and Qwen3-VL-4B — +6.34 on OmniDocBench, +7 on OCRBench, −4.68 on OCRBenchv2 English, +1.64 on OCRBenchv2 Chinese, −2.1 on DocVQA, +12.2 on CharXiv_DQ — demonstrates that OCR-specialized training provides large gains on domain-specific tasks while causing minor regressions on general VQA. This ablation is uncontrolled (the models were trained on different total token budgets and data distributions) but provides a useful reference point for the specialization-generalization trade-off.

Quantization robustness (Table 8): W8A8 quantization provides a 2× throughput improvement over W16A16 (1.024 vs. 0.503 PPS) with what the paper terms "negligible accuracy degradation." No accuracy numbers are reported for W8A8, so the claim of negligible degradation is unsubstantiated in the paper. This is a significant omission for a deployment-focused finding.

Specialized OCR model performance on general OCR and understanding benchmarks (Tables 5, 6): The paper's methodology note explains that for general OCR benchmarks and document understanding, "since specialized OCR models typically do not report these metrics, we integrate their OCR outputs in our evaluation environment." This functions as a cross-architecture ablation: pipeline systems' text extraction outputs, when fed to the same downstream evaluation pipeline or LLM, produce dramatically lower scores than end-to-end models on tasks requiring visual reasoning. The near-zero CharXiv scores for all two-stage systems (Table 6) are not a robustness check per se but serve as a powerful negative result demonstrating the cost of discarding visual context.

Missing ablations: The paper does not ablate several design choices that would strengthen the empirical case:

• Tile count and resolution: No experiments vary the maximum tile count (16) or tile size (448×448). Would 8 tiles suffice? Would 32 tiles improve small-font recognition?
• Coordinate special tokens: No ablation compares coordinate special tokens to plain digit encoding to quantify the claimed 50% output length reduction's impact on accuracy (beyond the latency claim).
• Layout taxonomy granularity: No head-to-head training comparison between the 25-category PaddleOCR-VL taxonomy and the 4-category MinerU 2.5 taxonomy. The paper argues for fine granularity's benefits but provides no controlled evidence — the taxonomy evaluation is only on detection accuracy of the teacher models, not on downstream student model performance.
• Revision or iterative refinement: No experiments test whether Layout-as-Thought benefits from iterative revision (generating layout, then response, then revising) or whether the one-pass layout-then-response approach is optimal.
• Thinking token position: No exploration of whether the thinking trigger should be prepended (as implemented) or could be inserted at different positions, or whether the model could learn to autonomously decide when to activate layout reasoning.
• KIE with vs. without Layout-as-Thought: No results on whether Layout-as-Thought improves KIE accuracy, despite KIE being a task where spatial reasoning should matter significantly (field locations on forms are inherently spatial).

Critical Assessment

Does Qianfan-OCR achieve state-of-the-art among end-to-end models on OmniDocBench v1.5?

Yes, decisively. The 93.12 score on OmniDocBench v1.5 (Table 4) ranks first among all end-to-end models by a margin of 2.03 points over the next-best (DeepSeek-OCR-v2 at 91.09). The advantage appears across all six sub-metrics, with particularly large gaps in table-related metrics (TableTEDs, TableTEDss). This claim is well-supported.

Does Qianfan-OCR narrow the gap with pipeline systems?

Yes, but with an important caveat. The 1.38-point gap with PaddleOCR-VL 1.5 (93.12 vs. 94.50) represents substantial progress over the previous best end-to-end result (DeepSeek-OCR-v2 at 91.09), narrowing the gap from 3.41 to 1.38 points — a ~60% reduction. However, the paper does not establish whether this gap is statistically significant. OmniDocBench v1.5's test set size is not stated in the paper; if it is in the range of 500–1,000 documents (typical for similar benchmarks), a 1.38-point difference on a composite metric may fall within benchmark noise, especially given that no confidence intervals or variance estimates are reported. The claim of "narrowing the gap" is supported directionally, but the magnitude of the remaining gap and its reliability are unclear.

Furthermore, PaddleOCR-VL 1.5 uses a 0.9B-parameter model — less than a quarter of Qianfan-OCR's 4B parameters. The paper presents this as evidence that end-to-end models are approaching pipeline accuracy, but a skeptic could invert the narrative: the best pipeline system achieves higher accuracy with a fraction of the parameters, suggesting that pipeline architectures are more parameter-efficient for pure recognition tasks. The paper does not engage with this counterargument.

Does Layout-as-Thought bridge the functionality gap for spatial grounding?

Partially, with demonstrated benefits conditional on document complexity. The Layout-as-Thought mechanism does produce explicit bounding boxes, element types, and reading order — the functionality that pipeline users need. The per-sample analysis (Figure 4) establishes that this spatial reasoning provides accuracy benefits on structurally complex documents. However, the evidence has several limitations:

1. Validation limited to one benchmark. Layout-as-Thought is evaluated only on OmniDocBench v1.5 for document parsing. Its effectiveness on KIE, document QA, chart understanding — tasks where spatial grounding would plausibly matter most — is explicitly unexamined (Section 7).

2. Aggregate accuracy penalty. The think mode underperforms no-think overall (92.64 vs. 93.12), meaning that naive activation across all documents would degrade average performance. Users must know when to activate it — a deployment complication the paper acknowledges but does not solve with an automatic activation criterion.

3. The mechanism's output quality is unevaluated. The paper shows an example of Layout-as-Thought output (Figure 5) but provides no quantitative evaluation of the layout analysis itself: bounding box IoU with ground truth, element type classification accuracy, reading order correctness. Without these metrics, we cannot assess whether the layout analysis is accurate enough to serve as a replacement for pipeline detection modules in applications that require spatial grounding.

4. The practical claim of "bridging the functionality gap" is stronger than the evidence supports. Pipeline users who need bounding boxes for downstream applications (e.g., redaction, spatial search, structured extraction with positional constraints) require accurate localization. The paper provides no evidence that Layout-as-Thought's bounding boxes are accurate enough for these use cases. The claim is demonstrated at the level of "the model can produce layout analysis" but not at the level of "the layout analysis is reliable enough for production use."

Does Qianfan-OCR outperform two-stage OCR+LLM pipelines on document understanding?

Yes, and the evidence is overwhelming. Table 6 shows two-stage systems achieving zero accuracy on CharXiv (both descriptive and reasoning), 2.0–17.2 on ChartBench (vs. 85.9), 7.0–56.8 on ChartQA (vs. 88.1), and 55.9–67.1 on DocVQA (vs. 92.8). These are not marginal differences — they represent a qualitative capability threshold where pipeline architectures fail entirely on tasks requiring visual-spatial reasoning. The finding is robust across four different OCR backends (PaddleOCR-VL, MinerU2.5, Dotsocr, DeepSeek-OCR), ruling out the possibility that the degradation is specific to a particular pipeline implementation.

However, the two-stage baseline has a confound: the downstream LLM is Qwen3-4B — the same model that serves as Qianfan-OCR's language backbone. This makes the comparison somewhat asymmetrical: Qianfan-OCR processes visual features directly (through the adapter), while the two-stage system feeds extracted text to the same language model without visual conditioning. The large performance gap could be attributed to either (a) the preservation of visual context in the end-to-end architecture, or (b) the end-to-end model's OCR-specialized training on the exact tasks being evaluated. The paper's design cannot cleanly separate these factors: an ideal control would be an end-to-end model trained with the same OCR specialization but forced to process only extracted text (somehow disabling visual features at inference), which would isolate the contribution of visual context preservation. In practice, this control is impossible, but the paper does not acknowledge the confound or discuss alternative explanations for the performance gap.

Does Qianfan-OCR achieve the highest KIE scores overall?

Yes, with a meaningful margin. The 87.9 overall mean across five KIE benchmarks (Table 7) represents a 3.7–8.7 point advantage over large commercial and open-source models. The result is particularly notable because Qwen3-VL-235B-A22B — with 50×+ more activated parameters — scores lower (84.2). The cross-lingual robustness (Chinese KIE: 82.3 vs. 62.9 for the 235B model) suggests the multilingual OCR data pipeline provides genuine benefits.

However, the KIE results come with an important caveat about benchmark composition. The paper reports normalized scores (0–100 scale) but does not describe the normalization procedure. Different KIE benchmarks have different native metrics (accuracy, F1, exact match), and the normalization methodology could affect relative rankings — especially if some models are evaluated under different protocols than others. The paper does not provide per-benchmark raw scores, only normalized scores, making it impossible to verify the normalization or assess whether specific benchmarks drive the overall mean.

Additionally, the KIE baselines are limited: Gemini-3.1-Pro, Gemini-3-Pro, Seed-2.0, Qwen3-4B-VL, Qwen3-VL-235B-A22B. Missing are comparisons with specialized KIE systems (layout-aware extraction models, form parsing systems) that might achieve higher accuracy on specific document types. The paper's claim of "highest average score" is accurate for the compared set but does not establish superiority over all possible approaches.

Does the multi-stage training recipe provide genuine benefits?

Yes, with ablation evidence, but with a gap between the ablated model and the final model. The ablation study (Table 2) convincingly shows that each training stage contributes to performance on the 8B model: Stage 2 provides the largest gain (+12.10 over Stage 1→Stage 4), Stage 3 provides incremental improvement (+0.92 over Stage 2→Stage 4), and the mixture of OCR-specific and general data in Stage 3 outperforms pure domain data. The finding that Stage 2 is essential and cannot be replaced by more Stage 3 data is methodologically important.

However, all ablation results are on the 8B model, not the final 4B Qianfan-OCR. The paper asserts that "the consistency of improvements across training stages suggests that this progressive recipe generalizes across model scales within the same architectural family," but this is an extrapolation, not a demonstrated fact. The 4B model's training recipe uses 70/30 domain-to-general split in Stage 3 (vs. 50/50 in the ablation), and the Stage 2 data composition for the 4B model explicitly differs from the ablation configurations (the 4B Stage 2 uses a specific 45/25/15/15 Document/Scene/Caption/Specialized split vs. the ablation's General or OCR+General configurations). The paper does not re-ablate these choices on the 4B model, so the final model's training configuration is based on extrapolation from the 8B results rather than direct validation.

Missing experiments that would strengthen the paper

Several experiments would substantially improve the empirical case:

1. Layout-as-Thought on KIE and document understanding benchmarks. Given that Layout-as-Thought is the paper's primary architectural innovation, its evaluation on only OmniDocBench is a significant limitation. KIE tasks — where knowing the spatial location of fields on a form is inherently useful — are the most natural application. Document QA, where questions like "what is the value in the bottom-right cell?" require spatial reasoning, would also benefit. Without these results, the paper cannot claim that Layout-as-Thought bridges the functionality gap for applications beyond document parsing.

2. Quantitative evaluation of Layout-as-Thought output quality. Reporting bounding box IoU, element type classification accuracy, and reading order accuracy would answer the question: is Layout-as-Thought a viable replacement for pipeline detection modules, or is it too inaccurate for production spatial grounding?

3. Accuracy numbers for W8A8 quantized model. The throughput comparison (Table 8) is a deployment-focused result, but without accuracy numbers for the quantized variant, the claim of "negligible accuracy degradation" is unsubstantiated. A simple OmniDocBench evaluation of the W8A8 model would close this gap.

4. Ablation of the 25-category layout taxonomy vs. coarser alternatives. The paper argues at length for fine-grained labels but never trains a model with coarse labels to demonstrate that fine granularity actually improves downstream performance. The current evidence is only about teacher model detection accuracy, not student model capability.

5. Statistical significance for key comparisons. The OmniDocBench gap (1.38 points vs. PaddleOCR-VL 1.5), the OlmOCR gap (0.2 points vs. PaddleOCR-VL), and the KIE gaps all lack variance estimates. Without these, we cannot assess whether the reported differences are reliable.

6. Difficulty-stratified KIE results. The KIE benchmarks likely contain documents of varying complexity (simple receipts vs. complex multi-page invoices). Breaking out performance by difficulty — analogous to the layout entropy analysis for OmniDocBench — would reveal where Qianfan-OCR's advantages are concentrated and whether Layout-as-Thought would help on spatially complex KIE documents.

7. Compute-matched comparison with pipeline systems. The paper claims end-to-end architectures can be "competitive" with pipeline systems, but the OmniDocBench leader shows a 0.9B model (PaddleOCR-VL 1.5) outperforming a 4B model (Qianfan-OCR) on accuracy. An apples-to-apples comparison training an end-to-end model at 0.9B scale, or a pipeline system at 4B scale, is not provided. The claim of "competitiveness" is true in absolute accuracy terms but obscures the parameter efficiency gap.

6. Limitations and Trade-offs

Limitation 1: The Performance Ceiling of End-to-End Architectures Relative to Pipeline Systems Remains an Open Question

The assumption or constraint. The paper positions Qianfan-OCR as evidence that end-to-end architectures are closing the gap with pipeline systems on structured document parsing, but the central assumption — that further scaling of end-to-end training will eventually match or surpass heavily optimized pipeline architectures — is untested. The paper explicitly acknowledges this as an open question (Section 7):

"As a pioneering attempt at end-to-end OCR, the ultimate performance ceiling of purely end-to-end architectures remains an open question — future work should systematically explore architectural innovations, training strategies, and data scaling laws to determine whether end-to-end models can fully match or surpass heavily optimized pipeline systems."

This is not a trivial acknowledgement; it identifies a fundamental uncertainty about whether there exists an inherent structural disadvantage to end-to-end architectures that no amount of scaling can overcome.

The consequence. The practical stakes are high for organizations deciding between architectures. Pipeline systems benefit from decades of specialized optimization on detection, recognition, and assembly as separate sub-problems — each module can be independently improved, and progress on any sub-problem (e.g., better table detection, better formula recognition) directly translates to overall system improvement. End-to-end models must solve all sub-problems simultaneously within a single set of parameters. If there exists a fundamental trade-off — where optimizing for table structure parsing degrades text transcription accuracy or vice versa — then the end-to-end ceiling may be permanently lower than the pipeline ceiling, regardless of data scale or training budget. The paper provides no evidence either way on this question; it demonstrates progress but not whether the progress will continue or asymptote.

What evidence exists in the paper. The primary evidence for the ceiling uncertainty is the persistent gap with the top pipeline system: PaddleOCR-VL 1.5 achieves 94.50 on OmniDocBench v1.5 using a 0.9B-parameter model, while Qianfan-OCR achieves 93.12 with a 4B model (Table 4). The parameter efficiency gap — 4.4× more parameters for 1.38 lower points — is consistent with the hypothesis that pipeline architectures have a structural advantage that end-to-end models have not yet overcome. Conversely, the paper's evidence that the gap has narrowed from 3.41 points (DeepSeek-OCR-v2) to 1.38 points (Qianfan-OCR) is consistent with the hypothesis that continued scaling will close it entirely. Both interpretations are compatible with the data; the paper cannot distinguish between them because it provides no scaling law analysis — no experiments varying model size, data volume, or training duration to project whether the accuracy curve is converging toward the pipeline ceiling or asymptoting below it.

Mitigation status. The paper does not attempt to address this uncertainty. Section 7 lists "ultimate performance ceiling" as a limitation requiring future work, but proposes no concrete methodology for investigating it — no scaling law experiments, no architectural ablations comparing end-to-end vs. pipeline parameter efficiency, no analysis of whether specific sub-metrics (formulas, reading order) resist improvement more than others. The limitation is acknowledged transparently but left entirely for future investigation.

Limitation 2: The 4B Parameter Footprint Limits Deployment in Resource-Constrained Settings, With Unvalidated Quantization Trade-offs

The assumption or constraint. The paper targets a 4B-parameter model as a deliberate compromise between reasoning capability and deployment practicality. The authors state (Section 7):

"Although W8A8 quantization enables competitive throughput on GPU (Section 6.5), Qianfan-OCR's 4B parameter footprint limits deployment in resource-constrained environments such as edge devices and CPU-only servers — future work should explore knowledge distillation and pruning to develop compact variants (1B–2B parameters) suitable for broader deployment scenarios."

The paper assumes that model compression techniques (quantization, distillation, pruning) can reduce the deployment burden with acceptable accuracy loss, but provides no evidence for this assumption beyond a single throughput measurement of the W8A8 quantized variant.

The consequence. For practitioners considering deployment, the missing piece is the accuracy-cost trade-off curve for compressed variants. The 4B model with W16A16 precision requires approximately 8 GB of VRAM for parameters alone (4B parameters × 2 bytes for FP16), plus KV cache memory (~82 MB for the GQA-optimized 32K context window), plus activation memory for the batch. On a single A100 (40 GB or 80 GB variant), this is feasible. On edge devices (Jetson Orin with 32 GB unified memory, mobile phone NPUs with 8–16 GB), or CPU-only servers where memory bandwidth is a hard constraint, the 4B footprint may be prohibitive regardless of throughput. The paper's W8A8 quantization result (1.024 PPS, Table 8) demonstrates that quantization preserves throughput, but the paper reports no accuracy numbers for the quantized model. The claim of "negligible accuracy degradation" is unsubstantiated. A practitioner deploying W8A8 cannot know whether they are accepting a 0.5-point or 5-point accuracy penalty, making the deployment decision speculative rather than data-driven.

The competitive positioning with pipeline systems further complicates this assessment. PaddleOCR-VL achieves 94.50 on OmniDocBench with a 0.9B-parameter model — less than a quarter of Qianfan-OCR's parameters. For a deployment where model size is the binding constraint (e.g., on-device inference, memory-limited cloud instances), the pipeline system's parameter efficiency may dominate any architectural advantages of the end-to-end approach. The paper's throughput results (Table 8) show that the quantized 4B model achieves throughput comparable to the 0.9B pipeline system, but throughput and parameter footprint are distinct constraints — a GPU with 8 GB VRAM can run a 0.9B model but not a 4B model, regardless of throughput.

What evidence exists in the paper. Only Table 8 provides throughput measurements for W16A16 (0.503 PPS) and W8A8 (1.024 PPS) on a single A100. No accuracy evaluation is reported for the quantized model. No latency measurements (time-to-first-token, end-to-end page processing time) are reported. No experiments with distilled or pruned smaller variants exist. The paper does not compare against pipeline systems at matched parameter counts — there is no end-to-end model trained at 0.9B to determine whether the accuracy gap with PaddleOCR-VL 1.5 is due to the architectural paradigm or simply to model scale.

Mitigation status. The paper identifies the limitation (Section 7) and suggests knowledge distillation and pruning as future work directions, but provides no results. The throughput comparison (Table 8) partially mitigates concerns about inference speed but does not address the fundamental parameter footprint constraint for resource-constrained environments.

Limitation 3: Layout-as-Thought Is Validated Only on OmniDocBench for Document Parsing, Leaving Its Applicability to Other Tasks Unexplored

The assumption or constraint. Layout-as-Thought is the paper's primary architectural innovation — the mechanism that recovers spatial grounding within the end-to-end paradigm and that the paper claims bridges the functionality gap with pipeline systems. Yet its evaluation is confined entirely to OmniDocBench v1.5 for document parsing. The paper acknowledges this explicitly (Section 7):

"The current Layout-as-Thought mechanism has only been validated on OmniDocBench v1.5 for document parsing, where it shows targeted benefits on structurally complex pages. Its effectiveness on other tasks — such as key information extraction, document QA, and chart understanding — remains unexplored."

The implicit assumption is that layout reasoning transfers across tasks: if the model can reason about spatial structure for document parsing, it should be able to use similar reasoning for KIE (locating specific fields), document QA (identifying which region contains the answer), and chart understanding (mapping visual elements to data semantics). This assumption is plausible but untested.

The consequence. For a practitioner evaluating whether to adopt Qianfan-OCR, the Layout-as-Thought mechanism is simultaneously the most novel capability and the least characterized. Several critical questions are unanswered:

• Does Layout-as-Thought improve KIE accuracy? KIE tasks require mapping field names ("Total Amount," "Invoice Date") to their spatial locations on a form. The model must identify that the value "1,247.50" next to the label "Total" is the extraction target, not a nearby unrelated number. Layout reasoning — identifying that these two elements are spatially adjacent and belong to the same semantic group — should theoretically help. But without empirical evidence, a practitioner cannot know whether the Layout-as-Thought's layout analysis is accurate enough at the granularity of individual form fields (which are typically much smaller and more tightly packed than document-level elements like paragraphs or figures).

• Does Layout-as-Thought improve document QA? Questions like "What is the value in the third row, second column?" or "Who signed the document?" require spatial reasoning. Layout-as-Thought provides explicit bounding boxes and element types, which could theoretically be used by the response generation to focus attention on the relevant document region. But whether the model actually uses layout information this way — or whether it ignores the self-generated layout analysis in favor of the raw visual features — is unknown without task-specific evaluation.

• Does the layout reasoning format generalize? The current implementation produces a rigid format (fixed <box>, <label>, <brief> structure). For some tasks, different spatial information might be relevant — for chart understanding, the model might need axis ranges rather than bounding boxes; for document QA, it might need content summaries of specific regions rather than the full page layout. The paper does not investigate whether the current format is flexible enough for these use cases.

Beyond the task-specific questions, there is a deployment consequence: the paper provides no automatic criterion for activating Layout-as-Thought. The per-sample entropy analysis (Figure 4) reveals that think mode helps on structurally complex documents but hurts on simple ones. In production, a user or system must decide, per-document, whether to prepend thinking. The paper suggests that "users should decide whether to enable the thinking mode based on the layout complexity of their target documents" but provides no automated complexity estimator, no decision threshold, and no method for dynamically activating layout reasoning mid-generation. A naive deployment that always enables thinking would degrade average performance (92.64 vs. 93.12 on OmniDocBench), while one that always disables it would forgo the gains on complex documents. The optimal strategy — estimating complexity first, then deciding — is not developed.

What evidence exists in the paper. The Layout-as-Thought evaluation comprises:

• Aggregate OmniDocBench scores (92.64 think vs. 93.12 no-think overall)
• Per-metric breakdown (TextEdit, FormulaCDM, TableTEDs, TableTEDss, R-orderEdit)
• The layout label entropy analysis (Figure 4) showing conditional benefits on complex documents
• A single qualitative example (Figure 5) showing the format of layout output on a math exam paper

No quantitative evaluation of the layout analysis itself (bounding box accuracy, element type classification accuracy, reading order correctness) is provided. No results on KIE, document QA, or chart understanding with Layout-as-Thought are reported.

Mitigation status. The paper acknowledges the limitation (Section 7) and describes future work directions: "future work should integrate these layout elements more naturally into the reasoning process, allowing the model to flexibly invoke spatial reasoning when needed rather than producing a fixed-format layout dump." It also proposes reinforcement learning as a mechanism for achieving task-adaptive layout reasoning. However, no empirical steps toward these goals are taken in the current paper.

Limitation 4: The Training Data Synthesis Pipelines Rely on Teacher Models With Undisclosed Failure Modes

The assumption or constraint. The six data synthesis pipelines (Section 3.2) form the foundation of Qianfan-OCR's capabilities, generating the training data for all four training stages from document parsing through chart understanding. These pipelines critically depend on teacher models — PaddleOCR-VL for document parsing and layout annotation, VLM-based description generation for chart understanding, DeepSeek models for instruction rewriting, multiple internal models for KIE and table extraction — whose predictions serve as ground truth for training Qianfan-OCR. The paper states:

"We construct an automated pipeline that converts document images into structured Markdown using PaddleOCR-VL (Cui et al., 2025b) for layout detection and content recognition."

The implicit assumption is that teacher model errors are either sufficiently rare to not materially affect student model performance, or are caught by the quality control mechanisms (multi-model agreement, rule-based filtering, consistency validation) before entering the training data. This assumption is never tested.

The consequence. Teacher model errors in the training data create a supervision ceiling: Qianfan-OCR can never be more accurate than its teachers on the specific documents and element types where teacher errors propagate into the training data. This is a form of annotation bias that is particularly pernicious because it is systematic rather than random — teacher models make correlated errors (e.g., PaddleOCR-VL may systematically misclassify certain element types, or underperform on specific document layouts), and these correlated errors become baked into the student model's learned behavior.

Specific failure modes that could arise:

• Layout taxonomy errors. If PaddleOCR-VL systematically confuses abstract with content on certain paper layouts, Qianfan-OCR will learn to make the same confusion, and no amount of scaling will correct it — the model is optimizing toward a flawed target.
• Table structure errors. The table pipeline uses consistency validation between two parsers, but if both parsers share a common failure mode (e.g., both fail on tables with complex merged cell patterns), inconsistent but both-wrong annotations may pass the consistency check and enter training.
• Chart understanding errors. The chart synthesis pipeline uses VLMs to generate visual descriptions and reasoning QA pairs. If the VLM hallucinates data values, misidentifies chart types, or produces internally inconsistent descriptions, these errors become training targets. The paper does not specify which VLM generates these descriptions or evaluate its accuracy on chart description tasks.
• KIE errors. The KIE pipeline uses "multi-model collaborative labeling" to address hallucination, but the paper provides no quantification of remaining hallucination rates. If teacher models hallucinate field values — generating plausible-sounding but incorrect extractions — these hallucinations become ground truth.

The training data volume (2T+ tokens across all stages) makes manual verification infeasible. The automated quality control mechanisms (consistency validation, rule-based filtering, cross-model verification) reduce but cannot eliminate annotation errors. The paper provides no estimate of residual error rates in the final training corpus, making it impossible to assess how close the student model is to the supervision ceiling.

What evidence exists in the paper. The paper provides:

• A comparison of teacher model label granularity (PaddleOCR-VL vs. MinerU 2.5) arguing for PaddleOCR-VL's superiority (Section 3.2)
• A mention that PaddleOCR-VL "achieves consistently higher detection accuracy" on a multi-type document layout benchmark (Section 3.2) — but no accuracy numbers are reported
• Descriptions of quality control mechanisms (consistency validation for tables, multi-model agreement for KIE) without quantification of their effectiveness
• No evaluation of teacher model accuracy on the specific data distributions used for synthesis
• No comparison of Qianfan-OCR's performance against its teacher models on the same documents, which would reveal whether the student has approached or exceeded the teacher's accuracy

Mitigation status. The paper does not acknowledge the teacher model dependency as a limitation. The quality control mechanisms are described as pipeline components but not validated. The paper frames the data synthesis pipelines as a strength — "large-scale, high-quality training data" — without critically examining the "high-quality" claim. Future work could address this by benchmarking teacher models on held-out data, reporting annotation error rates, or conducting human evaluation of training data quality, but none of these are performed in the current paper.

Limitation 5: All Evaluations Are on a Single Model Family With No Cross-Architecture Generalization Evidence

The assumption or constraint. The paper's findings — the effectiveness of multi-stage progressive training, the value of fine-grained layout taxonomy, the difficulty-dependent benefits of Layout-as-Thought, the superiority of end-to-end architectures over two-stage pipelines on visual-spatial reasoning tasks — are established entirely on the Qianfan-VL architectural family. The ablation study (Table 2) uses Qianfan-VL-8B, the final model is a 4B variant of the same architecture, and all comparisons use the same vision encoder (Qianfan-ViT), adapter (two-layer MLP), and language model backbone (Qwen3-4B). The paper states:

"The consistency of improvements across training stages suggests that this progressive recipe generalizes across model scales within the same architectural family."

This claim is about within-family generalization, but the paper's broader claims — about the viability of end-to-end architectures, the value of Layout-as-Thought, the catastrophic failure of two-stage pipelines — are implicitly presented as general findings about document intelligence rather than as properties of a specific model implementation.

The consequence. A practitioner using a different vision-language architecture (LLaVA, InternVL, LLaMA-Vision, different ViT backbones, different adapters, different language models) cannot assume that the training recipe, data mixtures, or Layout-as-Thought mechanism would transfer effectively. Several architectural choices could plausibly affect the results:

• Vision encoder resolution and capacity. Qianfan-ViT uses AnyResolution tiling with 448×448 patches and up to 16 tiles. A model using a lower-resolution encoder (e.g., standard 336×336 CLIP-based encoders common in LLaVA-style models) might not achieve the same OCR accuracy regardless of training data, because the visual features simply don't contain sufficient spatial resolution for small-font text recognition.
• Language model backbone. Qwen3-4B uses GQA with 32 query heads and 8 KV heads, RMSNorm, and a 32K native context window. A model using a different backbone (e.g., LLaMA-3.2-3B with different attention patterns or normalization schemes) might exhibit different training dynamics, different sensitivity to the progressive curriculum, or different capacity for layout reasoning.
• Cross-modal adapter design. The two-layer MLP adapter is a simple design; more sophisticated adapters (Q-former, perceiver resampling, cross-attention pooling) might compress visual information differently, potentially affecting the model's ability to retain fine-grained spatial detail needed for OCR.
• Coordinate special tokens. The 1,000-token coordinate vocabulary relies on the language model learning spatial representations from training data distribution. Whether this works as effectively with different tokenizer vocabulary sizes, different embedding dimensions, or different pretraining corpora is unknown.

The single-family evaluation also limits the strength of the architectural claims. The paper demonstrates that one specific end-to-end architecture can approach pipeline accuracy and dramatically outperform two-stage systems on visual-spatial reasoning. It does not demonstrate that end-to-end architectures in general have these properties. A skeptic could argue that Qianfan-VL happens to be a particularly strong architecture for OCR, and that other end-to-end designs might not achieve the same results.

What evidence exists in the paper. All experiments use Qianfan-VL architecture variants. The only cross-model comparison is with other published models (Qwen-VL, InternVL, DeepSeek-OCR, etc.) evaluated on the same benchmarks, but these are independent systems with different architectures, training data, and training recipes — they are not controlled comparisons varying only the architecture. The paper provides no experiments where the same training data and recipe are applied to different base architectures to test generalization.

Mitigation status. The paper does not acknowledge the single-architecture limitation. The ablation study on the 8B model provides within-family validation of the training recipe across model scales, but this does not constitute cross-architecture generalization evidence. Future work would need to replicate the findings on different vision-language architectures to establish generality.

Limitation 6: The Two-Stage Pipeline Baseline for Document Understanding Uses the Same Language Model as Qianfan-OCR, Creating a Confounded Comparison

The assumption or constraint. The document understanding comparison in Table 6 is the paper's primary evidence that end-to-end architectures dramatically outperform two-stage OCR+LLM pipelines on tasks requiring visual-spatial reasoning. The baseline systems are constructed by pairing specialized OCR models (PaddleOCR-VL, MinerU2.5, Dotsocr, DeepSeek-OCR) with Qwen3-4B as the downstream reasoning LLM. The paper describes this as:

"To simulate real-world usage scenarios, we employ a two-stage pipeline: first using specialized OCR models for text extraction, then feeding the extracted text to Qwen3-4B LLM for answer generation."

The assumption is that this baseline fairly represents the capability of two-stage architectures at comparable parameter counts and model quality. But there is a confound: the downstream LLM (Qwen3-4B) is precisely the language model backbone used by Qianfan-OCR. This means the comparison is not between an end-to-end architecture and a pipeline architecture with equivalent components, but rather between an end-to-end model that processes visual features directly and has been trained end-to-end on document understanding tasks and a pipeline system whose language model has not been fine-tuned for document understanding from extracted text.

The consequence. The large performance gaps in Table 6 — zero accuracy on CharXiv, 7.0–56.8 on ChartQA for pipelines vs. 88.1 for Qianfan-OCR — cannot be cleanly attributed to the preservation of visual context in the end-to-end architecture. They could equally be attributed to:

1. End-to-end task-specific training: Qianfan-OCR was trained on document understanding tasks (chart QA, document QA) as part of Stage 3 and Stage 4. The pipeline baseline's Qwen3-4B was not fine-tuned for these tasks — it receives extracted text and must answer questions zero-shot or few-shot. A fairer comparison would fine-tune Qwen3-4B on the same document understanding tasks using extracted text as input, to determine whether the performance gap is due to the input modality (visual features vs. extracted text) or the task-specific training.

2. OCR error propagation vs. visual context preservation: The pipeline systems' low performance could be due to OCR errors in the extracted text (misrecognized characters, lost formatting, garbled table structures) rather than the loss of visual spatial information. If Qwen3-4B receives garbled text from an imperfect OCR system, it may fail regardless of visual context. The paper does not measure OCR accuracy on the specific documents in the understanding benchmarks, making it impossible to distinguish OCR quality effects from architectural effects.

3. Prompt format mismatch: Pipeline systems feed plain text to Qwen3-4B. Qianfan-OCR receives the original document image and task prompt. If the task prompts for document understanding benchmarks contain visual references ("the chart above," "the highlighted region," "the bottom-right cell") that make sense only with visual input, the pipeline baseline is at a fundamental disadvantage that is not about visual context preservation but about the prompt's design assumptions about input modality.

The paper's conclusion — "for document intelligence tasks requiring joint visual and textual understanding, preserving visual context throughout the processing pipeline offers significant advantages over text-only intermediate representations" — is stronger than the evidence supports because the confounded comparison cannot isolate visual context preservation as the causal factor.

What evidence exists in the paper. Table 6 reports the raw performance numbers. The paper's interpretation attributes the gap to visual context: "chart structures, axis relationships, and data point positions – discarded during text extraction – are essential for answering." But the paper provides no ablation that would isolate this factor — no experiment where Qianfan-OCR receives only extracted text at test time, no fine-tuning of the pipeline LLM on document understanding data, no measurement of OCR accuracy on the understanding benchmarks' documents.

Mitigation status. The paper does not acknowledge this confound. The two-stage baseline construction is described as simulating "real-world usage scenarios," which is a pragmatic choice, but the paper does not discuss the limitations of this baseline or alternative explanations for the performance gap. A more rigorous comparison would require:

• Fine-tuning the downstream LLM on document understanding tasks with extracted text as input
• Reporting OCR accuracy on the benchmark documents to quantify error propagation
• Testing Qianfan-OCR's performance when forced to use only extracted text (e.g., by ablating the visual features at inference)
• Comparing against pipeline systems that use the same language model backbone and are trained on the same document understanding data

None of these controls are present in the paper, making the causal attribution of the performance gap to visual context preservation suggestive but not conclusive.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper does not introduce a new model architecture in the traditional sense — the ViT + MLP adapter + language model backbone is inherited from Qianfan-VL, and the 4B parameter scale is unremarkable in 2026. What it changes is the framing of what an end-to-end OCR system is responsible for providing. Before Qianfan-OCR, the field had implicitly accepted a dichotomy: pipeline systems provide spatial grounding (bounding boxes, element types, reading order) but sacrifice visual context for downstream understanding; end-to-end models preserve visual context but abandon spatial grounding as an explicit output. The choice between architectures was a choice about which capability to forgo. Layout-as-Thought demonstrates that this trade-off is not fundamental — an end-to-end model can produce explicit layout analysis as an optional reasoning behavior, controllable through the same prompt interface that governs the rest of its behavior.

This is a reframing, not a paradigm shift. The underlying technology (autoregressive VLM generating structured output) is unchanged. What shifts is the design philosophy: capabilities that were previously externalized into separate pipeline stages — layout detection, element type classification, reading order determination — can be internalized as learned reasoning subroutines that the model executes when prompted. The consequence is that the architectural debate between pipeline and end-to-end approaches is reframed from "which external components do you need?" to "which reasoning behaviors should the model learn?" This has methodological implications for how future OCR systems are designed, trained, and evaluated — the question is no longer whether a system has a layout detection module, but whether it can perform layout analysis when asked, and whether that analysis is accurate enough for downstream use.

The paper's second contribution to the landscape is empirically quantifying the cost of discarding visual context in a way that previous work had only qualitatively described. The CharXiv zero-accuracy result for two-stage OCR+LLM systems (Table 6) is not a marginal degradation — it is a binary capability threshold. When Qwen3-4B receives text extracted from a chart by a specialized OCR model, it cannot answer any question about that chart, because the spatial relationships, axis mappings, and data point positions that make the chart interpretable are absent from the text representation. This converts a qualitative architectural preference ("end-to-end models are better for understanding") into an empirical necessity: if chart understanding is a deployment requirement, pipeline architectures are structurally non-viable, regardless of how accurate their text extraction becomes. This finding should shift the burden of proof in the architecture debate — pipeline advocates must now demonstrate that their systems can perform visual-spatial reasoning tasks, rather than simply asserting that detection-first architectures are sufficient.

The paper also provides a new diagnostic for when spatial reasoning helps: the layout label entropy analysis in Figure 4. The finding that Layout-as-Thought helps on structurally heterogeneous documents but hurts on homogeneous ones establishes that spatial reasoning is not uniformly beneficial — it is a tool whose utility depends on the input. This diagnostic challenges the implicit assumption in previous work that "more spatial information is always better" and suggests that adaptive mechanisms (deciding per-document whether to invoke layout reasoning) are necessary for optimal deployment. The paper does not develop such an adaptive mechanism, but the entropy analysis provides the measurement methodology that future work can use to design and evaluate one.

The paper's third contribution is a methodological validation of multi-stage progressive training for OCR specialization. The ablation study (Table 2) provides controlled evidence that foundational OCR pretraining (Stage 2, 2T tokens) provides the bulk of capability, that domain-specific enhancement (Stage 3) provides incremental refinement, and that general data mixing acts as a regularizer preventing overfitting to narrow OCR patterns. This finding has practical significance beyond the specific model: it establishes that adapting a general VLM for OCR through instruction tuning alone — a common approach given the ease of constructing instruction datasets compared to pretraining corpora — will hit a capability ceiling that no amount of instruction data quality can overcome. The 2T-token Stage 2 budget, accounting for ~70% of total training tokens, quantifies the scale of pretraining investment required for competitive OCR performance. This is actionable guidance for organizations building OCR systems: budget for pretraining-scale compute, not just fine-tuning.

Follow-Up Research This Work Enables

1. Task-adaptive layout reasoning via reinforcement learning. The paper acknowledges that the current Layout-as-Thought implementation produces "a relatively rigid format via supervised fine-tuning" and that future work should allow the model to "flexibly invoke spatial reasoning when needed rather than producing a fixed-format layout dump" (Section 7). This is a concrete research direction: train the model to generate task-conditional layout reasoning using reinforcement learning, where the reward is downstream task accuracy (document parsing edit distance, KIE F1, chart QA accuracy). The key experiment: for a KIE task asking "What is the total amount?", does the model learn to produce a focused layout analysis attending only to the relevant document regions (the invoice total field and its spatial context) rather than enumerating all 60+ page elements? The paper's existing Layout-as-Thought training data provides supervised initialization; RL would fine-tune the layout generation policy to be task-adaptive. The strongest evidence would show that RL-trained layout reasoning achieves higher accuracy than SFT layout reasoning on downstream tasks while generating shorter, more focused layout traces — demonstrating that the model has learned to allocate its "thinking budget" efficiently.

2. Quantitative evaluation of Layout-as-Thought spatial grounding accuracy. The paper demonstrates that Layout-as-Thought produces bounding boxes, element types, and reading order, but provides no metrics on whether these outputs are correct. A follow-up study should evaluate Layout-as-Thought's layout analysis against ground-truth annotations on a dataset like PubLayNet or DocLayNet (which provide pixel-accurate bounding boxes and element type labels for document images). Key metrics: bounding box IoU (mean and distribution), element type classification accuracy (per-category and overall confusion matrix), and reading order Kendall's tau (comparing predicted element sequence to ground-truth reading order). This would answer the critical deployment question the paper leaves open: is Layout-as-Thought a viable replacement for pipeline detection modules in applications that require spatial grounding (redaction, spatial search, structured extraction with positional constraints), or is it too inaccurate for production use? The paper's OmniDocBench analysis provides a starting point — the same documents could be annotated with layout ground truth — but a dedicated layout analysis benchmark with diverse document types would be more informative.

3. Difficulty-adaptive activation of Layout-as-Thought. The paper's Figure 4 shows that Layout-as-Thought helps on high-entropy documents but hurts on low-entropy ones, but provides no mechanism for deciding per-document whether to activate it. A natural follow-up: train a lightweight "layout complexity estimator" — perhaps a small vision model or a regression head on top of the ViT features — that predicts the layout label entropy of an input document before the main model processes it. The estimator's output would gate the thinking token: activate Layout-as-Thought when predicted entropy exceeds a threshold, skip it otherwise. The research question: can a simple estimator achieve high enough accuracy that the gated system's aggregate performance exceeds both the always-think and never-think baselines? The OmniDocBench v1.5 dataset, with its per-page layout annotations, provides the training signal for the estimator. A successful result would convert Layout-as-Thought from a user-managed option into an automatically optimized inference strategy — directly addressing the "users should decide" limitation the paper acknowledges.

4. Scaling law analysis for end-to-end OCR architectures. The paper explicitly identifies the "ultimate performance ceiling of purely end-to-end architectures" as an open question (Section 7) but provides no methodology for investigating it. A follow-up study should train Qianfan-OCR variants at multiple scales (e.g., 0.5B, 1B, 2B, 4B, 8B parameters) with matched training data and recipes, and measure OmniDocBench and OlmOCR Bench accuracy as a function of model size, training tokens, and inference compute. The key finding would be whether the accuracy curve is converging toward the pipeline system ceiling (currently 94.50 on OmniDocBench) or asymptoting below it. If convergence is observed, the scaling law would predict the model size needed to match or exceed pipeline accuracy. If asymptoting is observed, the scaling law would identify the performance ceiling and potentially motivate architectural innovations (different vision encoders, attention mechanisms, or decoding strategies) that could break through it. This is expensive — training multiple 0.5B–8B models on 2T+ tokens each — but is the only way to answer the "open question" the paper raises about end-to-end architecture viability.

5. Cross-architecture replication of the training recipe. All findings in this paper are established on the Qianfan-VL architecture family. A replication study should apply the same four-stage progressive training recipe (with the same data synthesis pipelines, data mixtures, and hyperparameter schedules) to a different vision-language architecture — for example, a LLaVA-style model with a CLIP-based vision encoder and LLaMA-3 language backbone, or an InternVL-style model with a larger ViT. The research question: are the findings about multi-stage training effectiveness, OCR-vs-general data mixing, and Layout-as-Thought benefits architecture-specific, or do they generalize? A negative result — where the same recipe fails to produce competitive OCR performance on a different architecture — would be equally informative, revealing that Qianfan-VL's specific design choices (AnyResolution tiling, GQA attention, the particular adapter design) are load-bearing for OCR capability. A positive result would strengthen the paper's claims from "this specific model works" to "this training methodology works across architectures."

6. Layout-as-Thought for chart understanding and document QA. The paper's most significant unvalidated claim is that Layout-as-Thought could benefit tasks beyond document parsing. A follow-up should evaluate Layout-as-Thought on the chart understanding benchmarks (ChartQA, ChartQAPro, ChartBench, CharXiv) and document QA benchmarks (DocVQA, OCRVQA) where spatial reasoning should matter. The experiment: compare Qianfan-OCR with and without thinking tokens on these tasks, and analyze whether layout reasoning improves accuracy on spatially-grounded questions (e.g., "What is the value of the red bar in the third quarter?") more than on text-centric questions (e.g., "What is the title of this document?"). The paper's existing chart understanding training data (300K+ chart QA pairs from arXiv sources) provides the supervised signal for the model to learn task-relevant layout reasoning. If Layout-as-Thought provides significant gains on chart understanding, it would validate the paper's hypothesis that explicit spatial reasoning transfers across document intelligence tasks. If it provides no gains or degrades performance — as it does on low-entropy document parsing — it would establish a boundary condition and motivate the development of task-specific layout reasoning formats rather than a one-size-fits-all approach.

Practical Applications and Downstream Use Cases

1. Unified document processing pipelines in enterprise settings. The paper's throughput results (Table 8) demonstrate that a quantized 4B end-to-end model achieves 1.024 pages per second on a single A100 — comparable to the top pipeline system PaddleOCR-VL (1.224 PPS) despite having 4.4× more parameters. For an enterprise processing millions of heterogeneous documents monthly (contracts, invoices, reports, academic papers, forms), replacing a multi-stage pipeline with a single Qianfan-OCR instance eliminates the deployment complexity of orchestrating detection, recognition, and assembly stages. The concrete benefit: a single vLLM instance replaces what would otherwise require CPU-based layout detection, GPU-based recognition, and rule-based assembly — reducing operational burden and points of failure. The paper's W8A8 quantization makes this feasible on a single A100, and the competitive throughput means the switch does not impose a speed penalty. The Layout-as-Thought mechanism provides spatial grounding output for applications that need it (structured extraction, redaction, accessibility tagging), while the no-think mode handles homogeneous documents at maximum speed. The primary adoption barrier is the 4B parameter footprint for resource-constrained deployments — organizations with only CPU servers or edge devices would need the distilled 1B–2B variants the paper identifies as future work.

2. Chart and figure understanding in academic and scientific document processing. The CharXiv results (Table 6: 94.0 descriptive, 85.2 reasoning) and ChartQA results (88.1) establish Qianfan-OCR as capable of answering questions about academic charts at a level that dramatically exceeds two-stage OCR+LLM pipelines (which achieve zero). For academic search engines, literature review tools, and scientific knowledge bases that need to extract structured data from millions of published figures — identifying which drug showed what effect at which dosage, extracting numerical trends from line charts, answering questions about scatter plot correlations — Qianfan-OCR provides a single-model solution that preserves the visual encoding of data. The paper's chart understanding synthesis pipeline (300K+ training examples from arXiv LaTeX sources) means the model has been explicitly trained on the chart types common in scientific publishing. The practical deployment scenario: a scientific search engine indexes figures alongside text, allowing queries like "Show me all figures where the treatment group outperformed control with p < 0.01" — a capability that requires joint visual and textual understanding that pipeline architectures structurally cannot provide.

3. Cross-lingual document processing with strong Chinese performance. Qianfan-OCR achieves 60.77 on OCRBenchv2 Chinese text recognition (Table 5) and 82.3 on OCRBenchv2 KIE Chinese (Table 7) — substantially outperforming both same-scale general VLMs (Qwen3-VL-4B at 59.13 and 71.3 respectively) and much larger models (Qwen3-VL-235B-A22B at 62.9 KIE Chinese). For organizations processing Chinese-language documents — financial filings, government forms, legal contracts, medical records in China and other Chinese-speaking regions — this represents state-of-the-art accuracy in a deployable 4B package. The cross-lingual robustness (covering 192 languages through the multilingual OCR data pipeline) means the same model handles mixed-language documents (e.g., Chinese academic papers with English abstracts and formulas, bilingual contracts, international shipping documents) without switching between language-specific OCR engines. The practical benefit is eliminating the need for separate OCR systems per language or script — a single Qianfan-OCR instance handles the linguistic diversity that would otherwise require multiple specialized models.

4. Key information extraction from semi-structured documents. The paper's KIE results (Table 7: 87.9 mean across five benchmarks, surpassing Gemini-3.1-Pro by 8.7 points and Qwen3-VL-235B-A22B by 3.7 points) demonstrate that Qianfan-OCR can extract structured fields from invoices, receipts, ID cards, business licenses, and financial documents without the complex pipeline of detection + OCR + field mapping + rule-based validation that current industrial KIE systems require. The paper's KIE training data incorporates hard rule filtering (e.g., "unit price × quantity = total") and multi-model collaborative labeling, providing robustness against hallucinated extractions. The practical deployment scenario: an accounts payable system receives supplier invoices in diverse formats (PDF scans, photographs, email attachments). Qianfan-OCR directly extracts invoice number, date, line items, amounts, and tax identifiers into structured JSON, with the Layout-as-Thought mechanism optionally providing bounding boxes for each extracted field — enabling downstream verification workflows where a human reviewer can click on an extracted value to see its location on the original document. The single-model architecture eliminates the error propagation that occurs when a detection module misidentifies a field region and the downstream extraction module extracts from the wrong location.

When to Prefer This Method

The paper articulates a clear trade-off between three architectural approaches — end-to-end models with Layout-as-Thought, pipeline OCR systems, and general vision-language models — and provides the empirical evidence needed to make deployment decisions. The decision framework is:

Prefer Qianfan-OCR (or similarly designed end-to-end models with optional layout reasoning) when:

• The deployment task requires both structured document parsing and visual-spatial reasoning (chart understanding, document QA, or any task where two-stage OCR+LLM systems achieve near-zero accuracy as shown in Table 6). If CharXiv or ChartQA are in your evaluation suite, pipeline architectures are non-viable regardless of their OmniDocBench scores — the zero-accuracy result is a hard capability ceiling, not a marginal deficit.
• Cross-lingual robustness is critical, particularly for Chinese-language documents. Qianfan-OCR's Chinese KIE (82.3, Table 7) exceeds much larger models (Qwen3-VL-235B-A22B at 62.9), and the multilingual data pipeline covering 192 languages with writing-system-specific handling provides robustness that English-optimized models (Gemini-3.1-Pro, Seed-2.0) lack.
• Deployment simplicity matters more than squeezing out the last 1–2 points of structured parsing accuracy. The single-model serving (one vLLM instance) eliminates the orchestration complexity of multi-stage pipeline systems, and the W8A8 quantized throughput (1.024 PPS, Table 8) is competitive with the fastest pipeline system.
• The document distribution includes a mix of simple and complex documents, where an adaptive strategy (activating Layout-as-Thought on high-entropy pages) would maximize average accuracy. In this scenario, the conditional benefit demonstrated in Figure 4 can be exploited, though an automatic complexity estimator needs to be developed (the paper does not provide one).

Prefer pipeline OCR systems (PaddleOCR-VL 1.5 or similar) when:

• Pure recognition accuracy on structured documents is the sole requirement, with no downstream visual-spatial reasoning. PaddleOCR-VL 1.5 achieves 94.50 on OmniDocBench v1.5 with a 0.9B-parameter model (Table 4) — 1.38 points higher than Qianfan-OCR's 4B model with 4.4× fewer parameters. If your application is "transcribe this PDF to Markdown" and never needs to answer questions about charts or locate fields spatially, the parameter efficiency and accuracy lead of the top pipeline system are compelling.
• Deployment in CPU-only or edge environments where a 4B parameter model is infeasible. The 0.9B PaddleOCR-VL 1.5 can run in environments where Qianfan-OCR's 4B footprint — even with W8A8 quantization — exceeds memory constraints. The paper acknowledges this limitation (Section 7) and identifies distillation to 1B–2B as future work; until compact variants exist, pipeline systems remain the only option for resource-constrained deployment.
• Explicit layout analysis with verified accuracy guarantees is required, and the application cannot tolerate the unquantified localization errors of Layout-as-Thought. Pipeline detection modules have been benchmarked extensively on layout analysis accuracy (bounding box IoU, element type classification); Layout-as-Thought's layout analysis quality is unevaluated in the current paper, making it unsuitable for applications with strict spatial accuracy requirements (e.g., redaction of legally sensitive information from specific document regions).

Prefer general vision-language models (Qwen3-VL-4B or similar) when:

• The deployment task is general document understanding and VQA with no specialized OCR requirement. Qwen3-VL-4B outperforms Qianfan-OCR on DocVQA (94.9 vs. 92.8) and TextVQA (81.8 vs. 80.0) in Table 6, and on OCRBenchv2 English (60.68 vs. 56.0) in Table 5. The gaps are small (1.8–4.7 points) but consistent, and general VLMs benefit from ongoing scaling and improvement by their developers — a user adopting a general VLM gets continuous upstream improvements without maintaining OCR-specific training pipelines.
• Broad multimodal capabilities beyond document processing are needed — a general VLM handles natural images, scene text, diagrams, and non-document visual tasks that Qianfan-OCR's OCR-specialized training may have deprioritized (the paper does not evaluate non-document visual tasks, so the degree of specialization-related regression is unknown).
• The deployment compute budget is large enough to absorb the higher inference cost of general VLMs, and the marginal accuracy advantage on text-heavy VQA justifies the cost.

The paper's own positioning supports this differentiated decision framework: it does not claim universal superiority, acknowledges the remaining accuracy gap with pipeline systems on structured parsing (1.38 points on OmniDocBench, Section 6.1), and explicitly notes the regression on DocVQA and TextVQA relative to same-scale general VLMs (Section 6.3). The contribution is not to declare a winner in the architecture debate but to demonstrate that the boundaries between these approaches are narrowing — an end-to-end model can now be competitive on structured parsing, match general VLMs on document understanding, and provide the spatial grounding that was previously exclusive to pipeline systems — and that the choice between them is increasingly a matter of deployment constraints and task priorities rather than capability ceilings.