NVIDIA Nemotron Parse 1.1

ArXiv: 2511.20478

🎯 Pitch

An 885M-parameter model can match or beat systems over 10× its size at full document parsing—extracting text, tables, boxes, and semantics in one pass—while a 16× token-compressed variant loses almost no accuracy, proving that heavy visual compression is the key to production-speed document AI.

1. Executive Summary

NVIDIA introduces Nemotron-Parse-1.1, a lightweight 885M-parameter encoder-decoder document parsing and OCR model that improves upon its predecessor Nemoretriever-Parse-1.0 by unifying multiple extraction capabilities — general OCR, markdown formatting, structured table parsing, text extraction from pictures and diagrams, and semantic-aware bounding box prediction — within a single end-to-end architecture. Evaluated on public benchmarks including OmniDocBench, GOT, and RD-TableBench, the model achieves competitive accuracy against both pipeline-based and larger end-to-end systems, delivering a 0.958 F1 and 0.109 WER on an internal reading-order test set while outperforming Kosmos-2.5 and GOT across both plain OCR and markdown output modes. The paper also introduces Nemotron-Parse-1.1-TC, a variant using token compression via pixel-shuffle that reduces vision token length by 16× (from 3200 to 833 tokens), yielding a 20% inference speed improvement (4500 vs. 3800 tokens/second) with minimal quality degradation — for example, a 0.129 overall error rate on OmniDocBench compared to 0.131 for the full model — establishing that aggressive visual token reduction is viable for production document parsing, though the gains are most pronounced at the lower end of the model size spectrum where the decoder dominates latency.

2. Context and Motivation

The Core Problem: Document Parsing Requires Multiple Competing Capabilities

The central tension in document parsing is that real-world documents demand many things from a model simultaneously. A single page from a scientific paper, for instance, might contain multi-column body text with section headers, inline mathematical equations, a complex table with merged cells, a figure with a caption, footnotes, and page numbers — all of which must be extracted in correct reading order so that downstream systems (LLMs, retrieval pipelines, QA systems) can meaningfully consume the content.

Extracting any one of these elements well is challenging. Extracting all of them within a single system that runs fast enough for production use — that is the gap this paper addresses. The authors frame this explicitly in Section 1:

"Modern applications such as Large Language Models, retrieval systems, question-answering solutions, demand a richer representation, which includes layout, reading order, semantic classes (such as captions or footnotes), formulas, tables, and understanding of multi-column/multi-page structure."

This richer representation is not a luxury. For an LLM to answer "what is the main result in Table 3 of this paper," the document parser must (a) identify the table on the page, (b) extract its cell-level structure, (c) recognize that "Table 3" is a caption semantically linked to that table, and (d) preserve the reading order so the LLM knows the table's position relative to surrounding text. Any failure along this chain — a misidentified caption, a garbled merged cell, a table rendered as flat text — degrades the downstream system's answer quality.

Why This Problem Matters

The practical stakes are high for several reasons, some stated explicitly in the paper and others implicit in how document parsing fits into modern AI pipelines.

1. LLMs are only as good as the documents they ingest. The dominant paradigm for deploying LLMs on proprietary or domain-specific document collections is Retrieval-Augmented Generation (RAG): parse documents, chunk them, embed chunks, retrieve relevant chunks at query time, and feed them into the LLM as context. Every stage of this pipeline depends on the parser's output quality. If the parser scrambles reading order — placing a figure's caption before the paragraph it belongs to — the retriever may surface semantically incoherent chunks. If table cells are extracted as unstructured text, the LLM cannot reason about spreadsheet-like structures. The paper does not explicitly frame its work in RAG terms, but this is the primary downstream use case that gives document parsing its economic importance in 2025.

2. Cost and latency constrain deployment. Document parsing is often a preprocessing step applied to millions of pages in batch. A parser that takes 5 seconds per page on a high-end GPU is infeasible for processing billion-document corpora. This is where the paper's emphasis on a lightweight 885M-parameter architecture — and the token-compressed TC variant — becomes significant. The authors explicitly position Nemotron-Parse-1.1-TC as suitable "for large-scale batch processing, edge deployments, or interactive systems where rapid response times are critical" (Section 1). The 20% speed improvement from token compression (4500 vs. 3800 tokens/second on a single H100, Table 8) translates to roughly 5 pages per second for the TC variant, which is a throughput number that makes million-page processing economically viable.

3. The gap between pipeline and end-to-end approaches is narrowing but not closed. The paper identifies a fundamental tradeoff in current document parsers (Section 1):

"Pipeline solutions often rely on brittle multi-stage pipelines with each stage responsible for a subtask... achieving versatility at the cost of the lower throughput. At the same time, end-to-end models benefit from fast inference speeds while often not performing equally well on all subtasks associated with document extraction simultaneously."

Pipeline systems (like Dolphin, MinerU, or Marker) decompose document parsing into sequential stages: layout detection, OCR, table extraction, reading-order determination, formatting. This decomposition gives each stage clear responsibility and allows specialized components, but it introduces failure cascades — an error in layout detection propagates to every subsequent stage. It also incurs latency overhead from running multiple models sequentially and passing structured data between them.

End-to-end models (like Nougat, SmolDocling, GOT, or Nemotron-Parse itself) address the latency and cascade problems by predicting everything in a single forward pass, but they face the harder challenge of learning all subtasks simultaneously from heterogeneous training signals. Most end-to-end models before Nemotron-Parse-1.1 sacrificed quality on at least one dimension — often tables, formulas, or semantic classification — in exchange for speed. The paper's stated goal is to close this gap:

"we introduce Nemotron Parse 1.1... capable of extracting formatted text (Markdown/LaTeX), bounding boxes of text blocks, and semantic classes for each block while preserving the reading order." (Section 1)

Where Prior Approaches Fall Short

The paper identifies specific limitations across several axes, both explicit critiques and implicit comparison points visible in the benchmark tables.

Pipeline systems are accurate but slow and brittle. The OmniDocBench results in Table 4 show Dolphin achieving the best overall English score (0.356) among all systems — but it is a pipeline model. Marker achieves a table score of 0.609, best in its category. These numbers demonstrate that pipeline systems can deliver high accuracy on individual subtasks. However, the paper's summary observation that pipelines achieve "versatility at the cost of lower throughput" points to their core weakness: the latency and infrastructure complexity of chaining multiple models makes them unsuitable for high-volume processing. The paper does not report latency comparisons against pipeline systems, but this is a well-known limitation: each stage requires its own model inference, and many pipeline components (particularly OCR engines) are computationally intensive.

End-to-end models make task-specific compromises. Looking at Table 4, the pattern is clear:

• Nougat (Blecher et al., 2024) was a landmark end-to-end academic document parser, but its overall error score of 0.452 on OmniDocBench reflects significant weakness on text extraction (0.365) and formulas (0.488). It was trained predominantly on arXiv papers, limiting its generalization to other document types.
• GOT-OCR2.0 (Wei et al., 2024) has a poor overall score of 0.287, with particularly weak table performance (0.459). The paper's internal benchmark (Table 2) shows GOT scoring only 0.302 WER and 0.818 F1 in OCR mode and 0.259 WER / 0.879 F1 in markdown mode — substantially behind Nemotron-Parse's 0.109 WER and 0.958 F1.
• SmolDocling (Nassar et al., 2025) is perhaps the closest architectural comparison (an ultra-compact VLM at 392 vision tokens), and it achieves a competitive 0.493 overall score — but with significantly worse text extraction (0.262) than Nemotron-Parse-TC (0.055) and a reading order score of 0.227 vs. Nemotron-Parse-TC's 0.048. This illustrates the challenge: a small model can achieve good aggregate performance while having serious weaknesses in specific subtasks.
• Qwen2.5-VL-72B and InternVL3-78B are massive general-purpose VLMs (72B–78B parameters) that perform document parsing as one capability among many. Their OmniDocBench scores (0.214 and 0.218 respectively) show that scale alone does not guarantee document understanding — specialized training data and architecture matter more than raw parameter count, since these models are orders of magnitude larger than Nemotron-Parse (885M) yet underperform it on document-specific metrics.

The most revealing comparison in Table 4 is the vision token count column. DeepSeek-OCR models (Wei et al., 2025) achieve strong results with very low vision token counts (64–795 tokens), but their overall scores cluster around 0.123–0.386. Nemotron-Parse operates at 3201 tokens (a much higher resolution) and achieves 0.131 overall error — competitive with DeepSeek-OCR-Gundam (0.127 at 795 tokens) but with 4× the vision tokens. The TC variant at 833 tokens (close to DeepSeek-OCR's high end) matches the full model's performance (0.129 overall). This positions Nemotron-Parse as exploring a crucial design dimension: how many vision tokens are actually needed for document parsing, and can aggressive compression preserve quality?

Previous NVIDIA work set the stage but left gaps. The paper is explicit about being a successor to Nemoretriever-Parse-1.0 and Eclair (Karmanov et al., 2025). The abstract states that Nemotron-Parse-1.1 "advances the capabilities of its predecessor" across four dimensions: "general OCR, markdown formatting, structured table parsing, and text extraction from pictures, charts, and diagrams." It also "supports a longer output sequence length for visually dense documents." The details of what specific improvements were made relative to the predecessor are not enumerated in the paper — no ablation table comparing 1.0 to 1.1 is provided — so the reader must infer from benchmark results that the advances are in the quality of formatting (especially tables and formulas) and in the improved reading order of the TC variant, which "includes non-reading-order (floating) elements, i.e., Footnotes, Page-Footers, Tables, Pictures, and Captions within the natural ordering of the page" (Section 2.2.2).

Multilingual document parsing is underexplored in compact models. The paper's multilingual evaluation (Table 7) on 10,000 documents per language across 7 languages shows F1 > 0.96 across all languages, with notably strong performance on Chinese and Japanese (0.98 F1) in the scientific domain. This is achieved through a combination of machine-translated NVpdftex data and multilingual Wikipedia OCR data (Table 1). Most prior end-to-end OCR models focused primarily on English, with multilingual support being either absent or substantially weaker. The paper demonstrates that a relatively small training data investment in multilingual sources — specifically, applying machine translation to the NVpdftex pipeline output and incorporating Wikipedia data in 10 languages — can yield broad multilingual competence without fundamentally changing the model architecture.

Bounding box and semantic class prediction is a differentiator. Many OCR systems output only text. Nemotron-Parse outputs structured bounding boxes with semantic classes (Title, Section-Header, Text, List-Item, Formula, Table, Picture, Caption, Footnote, Page-Footer, Page-Header), which are critical for downstream tasks like layout-preserving chunking for RAG. The authors note this as a first-class capability, not an afterthought: the model is trained "jointly on heterogeneous datasets that provide different supervision signals (plain or formatted text, bounding boxes, and semantic classes)" (Section 2.2.1). The prompt-based interface allows users to request exactly the output format they need, from plain text only to the "maximal-information prompt" that includes formatted text, bounding boxes, and semantic classes. This flexibility means the same model can serve different downstream consumers — a search system might need only plain text, while a document viewer needs bounding boxes for highlight overlays.

How This Paper Positions Itself

Nemotron-Parse-1.1 positions itself as a practical, lightweight, broadly capable end-to-end OCR model that does not require choosing between speed and versatility. The positioning is communicated through several key design choices and rhetorical moves:

1. "Competitive, not dominant" as a deliberate tradeoff. The paper never claims state-of-the-art on any single benchmark. On OmniDocBench (Table 4), Dolphin achieves 0.356 overall vs. Nemotron-Parse's 0.131 (lower is better), and Marker achieves 0.296. On RD-TableBench (Table 6), Reducto achieves 90.2 vs. Nemotron-Parse's 85.8. On the GOT benchmark (Table 3), Gemini Flash 2.0 achieves 0.9915 F1 vs. Nemotron-Parse's 0.9785. The value proposition is not being #1 on any single metric — it is being consistently strong across all metrics while maintaining a small parameter footprint (885M) and high throughput. In other words, the model is designed to be "good enough at everything" rather than "best at one thing," which aligns with the production use case where documents contain heterogeneous content and the parser cannot be swapped based on page content.

2. Token compression as a practical knob, not a novel contribution. The TC variant's pixel-shuffle compression is not presented as a research innovation — it's a straightforward downsampling operation applied after the vision neck's convolutional compression. Its significance is empirical: it demonstrates that aggressive vision token reduction (16×) preserves quality well enough for production use, providing a 20% speed improvement. For a practitioner deciding whether to deploy Nemotron-Parse or Nemotron-Parse-TC, the benchmark tables show the tradeoff concretely: on OmniDocBench, TC loses 0.002 overall error points; on the internal test set (Table 2), TC loses 0.005 F1; on RD-TableBench (Table 6), TC loses 0.4 points of table similarity. These are small degradations that many production systems would accept in exchange for 20% higher throughput.

3. Open release as a community contribution. The paper emphasizes the release of model weights, the NIM container, the training data subset (as part of Nemotron-VLM-v2), and the NVpdftex generation pipeline. This is not incidental to the paper's contribution — for a document parsing model to be useful, practitioners need to be able to run it on their own documents. The public release makes the model a deployable artifact, not just a benchmark entry.

4. Training data diversity as the core enabler. While the architecture (RADIO encoder + mBART decoder) is a standard encoder-decoder design, the paper's energy is invested in describing the training data pipeline. Section 3.1 details the NVpdftex pipeline (LaTeX compilation with structured-output extraction), the DocLayNet augmentation strategy, the Common Crawl human-annotated data, the synthetic table and dense OCR data generation, and the multilingual Wikipedia processing. This reflects a conviction — shared across much of the modern OCR literature — that data quality and diversity matter more than architectural novelty for document parsing. The model architecture has 885M parameters; the training data spans millions of examples across 10+ languages, multiple formats (plain text, markdown, LaTeX), and multiple annotation types (text, bounding boxes, semantic classes). The paper positions its contribution as much in the data curation as in the model itself.

5. An evolutionary, not revolutionary, step. The paper does not claim to have solved document parsing. The abstract says it "advances the capabilities of its predecessor" — language that acknowledges incremental progress. The OmniDocBench results show it underperforms several pipeline and larger end-to-end systems. The table extraction results (Tables 5–6) show room for improvement on structured table understanding (TEDS of 81.3 on PubTabNet vs. the near-perfect scores achieved by some specialized table extraction systems). The paper's framing is that of a solid, practical system that pushes the state of lightweight end-to-end parsing forward by a meaningful increment, with particular strengths in reading order, compactness, and the integration of semantic class prediction into the OCR output format.

In sum, the paper addresses a real production need — fast, versatile document parsing that extracts structured, semantic-aware output — and positions its model as the best current option for practitioners who need all of these capabilities in a single, lightweight, deployable system, even if it is not the single best at any one capability.

3. Technical Approach

3.1 Reader Orientation

Nemotron-Parse-1.1 is an end-to-end vision-language model that takes a document image as input and directly outputs formatted text, bounding boxes, and semantic class labels — all in a single autoregressive generation pass. The problem it solves is that real-world documents contain heterogeneous elements (body text, tables, formulas, figures, footnotes) spread across complex layouts, and extracting all of these simultaneously with correct reading order from a single lightweight model has historically meant accepting poor performance on at least some element types; the solution is an 885M-parameter encoder-decoder transformer trained on a large and diverse blend of synthetic, public, and human-annotated data that covers the full spectrum of document extraction subtasks, combined with a prompt-based interface that lets the same model output different levels of information depending on the downstream need.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components that process information sequentially:

1. Vision Encoder (RADIO $\mathcal{E}$) — a ViT-H/16 (657M parameters) that maps the input document image $I \in \mathbb{R}^{3 \times H \times W}$ to a latent representation $Z \in \mathbb{R}^{N \times d}$, where $d$ is the hidden dimension and $N$ is the sequence length of visual tokens.

2. Vision Neck $\mathcal{N}$ — a convolutional downsampling module (1×4 horizontal kernels, stride 1×4) that compresses the visual token sequence spatially, reducing $N$ to 3200 tokens for a 1648×2048 input. For the TC variant, an additional pixel-shuffle operation further reduces this to 833 tokens (16× total compression). A summary token from RADIO is concatenated to the compressed sequence.

3. Language Decoder $\mathcal{D}$ — a 10-layer mBART decoder (256M parameters) with tied weights and no positional embeddings, which autoregressively generates text tokens $\{t_{P+1}, t_{P+2}, ..., t_L\}$ by conditioning on the compressed visual features $\mathcal{N}(Z)$ and the previously generated prompt-plus-output tokens. The model supports multi-token prediction via $m-1$ additional linear layers that predict blocks of $m$ tokens simultaneously.

4. Prompt Interface — three independent prompt tokens (<output_markdown>/<output_plain>/<output_no_text>, <predict_bbox>/<no_bbox>, <predict_classes>/<no_classes>) that condition the decoder to produce exactly the requested output format, enabling a single model to serve different downstream consumers with different information needs.

Information flows as follows: a document image enters the system → the RADIO vision encoder produces a grid of visual features → the vision neck compresses and reshapes these features (with pixel-shuffle compression added for the TC variant) → a summary token is concatenated → the compressed visual tokens are fed as cross-attention context to the mBART decoder → the decoder receives prompt tokens specifying the desired output format → the decoder autoregressively generates output text containing formatted content, bounding box coordinates, and semantic class labels in the specified interleaved format.

3.3 Roadmap for the Deep Dive

• First, the vision encoder and neck (Section 3.4.1), since all downstream processing depends on the quality and resolution of the visual features, and because the compression ratio is the key design parameter distinguishing the base model from the TC variant.
• Second, the language decoder architecture (Section 3.4.2), including the critical design choice to omit positional embeddings and the multi-token prediction mechanism, since these choices affect both sequence length generalization and inference speed.
• Third, the prompt interface and output format (Section 3.4.3), because the model's ability to handle heterogeneous training data and serve different downstream needs depends entirely on how prompts map to outputs.
• Fourth, the training data and data generation pipeline (Section 3.4.4), since data diversity — not architectural novelty — is the primary enabler of the model's broad capabilities, and the NVpdftex pipeline in particular is the core contribution to data quality.
• Fifth, the training procedure and multi-token training strategy (Section 3.4.5), including how teacher forcing is applied for multi-token prediction heads and why this also improves single-token inference accuracy.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and data engineering paper whose core idea is that a standard encoder-decoder architecture, when trained on a sufficiently diverse and carefully constructed data blend with a flexible prompt interface, can match or approach the performance of much larger models and multi-stage pipelines on document parsing while maintaining high throughput and a small parameter footprint.

3.4.1 Vision Encoder and Neck: From Pixels to Compressed Visual Tokens

The vision encoder $\mathcal{E}$ is initialized from RADIO (Ranzinger et al., 2024; Heinrich et al., 2025), which is itself a ViT-H/16 architecture (Dosovitskiy et al., 2021) with 657M parameters. The ViT-H/16 designation means it is a Vision Transformer using patch size 16×16 pixels ("/16") at the "Huge" scale, though the specific parameter count places it in the upper range of that category.

The encoder maps an input document image to a latent representation:

$Z = \mathcal{E}(I), \quad Z \in \mathbb{R}^{N \times d}$

where $I \in \mathbb{R}^{3 \times H \times W}$ is the input document image with 3 color channels, height $H$, and width $W$; $Z$ is the output grid of visual features; $d$ is the hidden dimension (the embedding size of each visual token, determined by the RADIO architecture and not explicitly stated in the paper); and $N$ is the number of visual tokens — the number of spatial positions in the feature grid, which for a ViT-H/16 is proportional to $(H/16) \times (W/16)$ plus any additional tokens (such as a CLS token in standard ViT architectures).

What this operation computes: It takes a raw document image — a grid of RGB pixel values — and transforms it through a series of self-attention and feed-forward layers into a grid of feature vectors, where each vector at position $(i,j)$ encodes the visual content of the corresponding 16×16 pixel patch. The output $Z$ is a compressed, semantically rich representation of the document's visual appearance: text strokes, table gridlines, figure regions, and layout boundaries are all encoded in these feature vectors.

Why this form: Using a pre-trained vision foundation model (RADIO) rather than training the encoder from scratch on OCR data is the standard modern approach — it leverages representations already trained on diverse visual data, which provides robustness to variations in fonts, colors, and document styles that an OCR-specific encoder might overfit to. The ViT architecture is chosen over convolutional alternatives because its global self-attention can model long-range layout relationships (e.g., connecting a footnote at the bottom of a page to its reference in the main text) without the limited receptive fields of CNNs. RADIO specifically is an agglomerative vision model trained to unify representations across multiple vision domains, which likely provides better generalization to diverse document types than a single-domain pre-training.

The vision neck $\mathcal{N}$ applies a convolutional downsampling operation to the encoder output:

$\mathcal{N}(Z): \mathbb{R}^{N \times d} \rightarrow \mathbb{R}^{N' \times d}$

The paper specifies that the neck consists of "horizontal convolutional kernels of size 1×4 and stride 1×4." This means the convolution kernel spans 1 token in the vertical (row) dimension and 4 tokens in the horizontal (column) dimension, and it moves with stride 1 vertically and stride 4 horizontally. For an input image of 1648×2048 pixels, the encoder produces some initial token count, and the neck reduces this to 3200 tokens (so $N' = 3200$ for the base model).

What this operation computes: It spatially downsamples the visual token grid in the horizontal direction only, reducing the sequence length by a factor of approximately 4 while preserving the vertical resolution. Since most documents are portrait-oriented (height > width) and contain text organized in horizontal lines, compressing horizontally more aggressively than vertically makes sense: the model needs high vertical resolution to distinguish text lines, but can tolerate coarser horizontal resolution within each line.

Why this form: The asymmetric (1×4) kernel design reflects a document-specific inductive bias. Text in documents flows left-to-right (or right-to-left) along horizontal lines, so the visual features along a horizontal row are more redundant than features along a vertical column (where line breaks and section boundaries create sharp transitions). Compressing horizontally more aggressively reduces the decoder's cross-attention cost (which scales with the number of visual tokens) while preserving the vertical resolution needed for accurate reading order and line-level text extraction.

Why 3200 tokens for a 1648×2048 image? Working backwards: a ViT-H/16 on a 1648×2048 image produces roughly $(1648/16) \times (2048/16) = 103 \times 128 = 13,184$ patches (plus any additional tokens). After the neck's 1×4 compression (which reduces the horizontal dimension by a factor of 4), this becomes approximately $103 \times (128/4) = 103 \times 32 = 3,296$ tokens — close to the stated 3200 (the discrepancy may come from exact patch grid dimensions, padding, or the concatenation of the RADIO summary token).

After the neck, "we additionally concatenate the summary token of RADIO to the sequence." RADIO, like many modern ViT architectures, includes a summary token (similar to a CLS token) that aggregates global image-level information. Concatenating this to the compressed spatial tokens gives the decoder access to both local visual features (for reading individual text segments) and a global representation (for understanding overall document structure).

Token compression for the TC variant. For Nemotron-Parse-TC, an additional pixel-shuffle operation is applied:

$\text{pixel-shuffle}: \mathbb{R}^{3200 \times d} \rightarrow \mathbb{R}^{833 \times d}$

Pixel-shuffle (also known as sub-pixel convolution or depth-to-space) is an operation that rearranges elements from the channel dimension into the spatial dimensions. Concretely, it takes groups of $r$ feature vectors along the spatial dimension and stacks their channel elements to produce a single feature vector with $r \times d$ channels at a coarser spatial resolution, then typically applies a linear projection back to $d$ channels. The paper states this achieves "a total of ×16 reduction" relative to the pre-neck token count, producing 833 tokens.

What this operation computes: It further compresses the 3200-token visual representation to 833 tokens by folding spatial information into the channel dimension and then projecting back to the original hidden dimension. This is a lossy compression step that discards fine-grained spatial details in exchange for a shorter sequence.

Why this form: The TC variant is designed for throughput-sensitive applications. The decoder's cross-attention cost scales quadratically with the number of visual tokens (each generated text token attends to all 3200 visual tokens for the base model). Reducing this to 833 tokens provides roughly a 3.8× reduction in cross-attention computation per generated token. Combined with the decoder's self-attention (which dominates at short sequence lengths but shares the cost), the paper reports a 20% end-to-end speed improvement (4500 vs. 3800 tokens/second, Table 8). The use of pixel-shuffle specifically — rather than, say, additional strided convolution — is a common design choice in efficient vision architectures because it preserves information density by reallocating spatial resolution into channel depth rather than simply discarding features.

A critical observation about the token count: Nemotron-Parse's 3201 visual tokens (the "3200" plus the summary token, as shown in Table 4) is substantially higher than most competing end-to-end models. SmolDocling uses 392 tokens, DeepSeek-OCR models range from 64 to 795 tokens, and GOT-OCR2.0 uses 256 tokens. The high token count reflects a deliberate design choice to preserve spatial detail — the model "sees" the document at higher resolution than its competitors. The TC variant at 833 tokens brings it into the same range as DeepSeek-OCR-Gundam (795 tokens), and the OmniDocBench results show that the TC variant achieves nearly identical performance (0.129 vs. 0.131 overall error), suggesting that 833 tokens may be close to the point of diminishing returns for document parsing — enough resolution to capture text and layout, but not so much that the decoder is overwhelmed with redundant spatial information.

3.4.2 Language Decoder: Autoregressive Generation with No Positional Embeddings

The decoder $\mathcal{D}$ uses an mBART (Liu et al., 2020) architecture reduced to 10 layers with tied weights (the same weight matrix is used for the input embedding and the output projection). It has 256M parameters, making it a relatively compact language model — roughly one-third of the total 885M parameters, with the remaining 628M residing in the vision encoder (657M) and neck/summary token overhead.

The decoder predicts text tokens autoregressively:

$P(t_i | \mathcal{N}(Z), t_{<i})$

where $\mathcal{N}(Z)$ is the compressed visual features from the encoder+neck (served as cross-attention context), $t_{<i} = \{t_1, t_2, ..., t_{i-1}\}$ are the tokens generated so far (including the prompt tokens $\{t_1, ..., t_P\}$), and $t_i$ is the next token to be predicted.

What this equation computes: For each generation step $i$, the decoder takes as input all previously generated tokens and the entire compressed visual representation of the document, and produces a probability distribution over the next possible output token. The model samples or greedily selects from this distribution to extend the output sequence.

Why mBART? mBART is a multilingual sequence-to-sequence architecture originally designed for neural machine translation. Its encoder-decoder structure is a natural fit for document parsing because the visual features serve as the "source language" representation (encoded by the vision tower), and the markdown/LaTeX/bounding-box output is the "target language" (decoded autoregressively). The 10-layer reduction (from standard mBART's 12 encoder + 12 decoder layers, though mBART variants vary) keeps the decoder lightweight while preserving enough capacity for the structured output task. Tied weights further reduce parameter count and provide a regularization effect — the same embedding space is used to represent both input tokens (the prompt and previously generated output) and output token predictions, encouraging consistency.

The critical design choice: no positional embeddings. Section 2.1.1 provides the rationale in detail. Standard transformer architectures add positional encodings (learned or sinusoidal) to token embeddings before the first layer so the attention mechanism — which is otherwise invariant to permutation — can distinguish token order. Nemotron-Parse's decoder omits these entirely.

The paper argues this is viable because "in causal decoder-only models the attention mask already provides positional cues: each token can only attend to preceding elements, which enables the model to infer its location in the sequence" (citing Kazemnejad et al., 2023; Zuo et al., 2025). Concretely: in a causal transformer, token $t_i$ can attend to tokens $t_1$ through $t_i$ but not to $t_{i+1}$ or beyond. This asymmetric attention pattern breaks permutation invariance — the model can learn that tokens attended to by many subsequent tokens are likely earlier in the sequence, and tokens that attend to many preceding tokens are likely later.

Why this matters for OCR specifically:

1. Sequence length generalization: Documents vary dramatically in length — a receipt might produce 100 output tokens, while a dense academic page might produce 10,000+ tokens. Models trained with learned positional embeddings are typically limited to the maximum sequence length seen during training (or require interpolation that can degrade quality at test time). By omitting positional embeddings entirely, Nemotron-Parse can in principle handle arbitrarily long output sequences without architectural modification. The paper states this explicitly: the NoPE approach allows "inference with significantly longer context lengths" than what was seen during training.

2. No interference with 2D spatial information: The visual tokens from the encoder already carry 2D spatial structure (each visual token corresponds to a specific region of the document page). If the decoder used 1D positional embeddings on the text tokens, these 1D position signals could conflict with the 2D layout information that the text is supposed to reference. For example, an output token at position 500 might correspond to text in the upper-left corner of the page, while an output token at position 501 might correspond to text in the lower-right corner (due to multi-column layout). A 1D positional embedding would encode them as "adjacent" while the 2D visual features encode them as "distant." Omitting positional embeddings removes this potential conflict.

3. Computational efficiency: Removing positional embedding parameters and the associated computations (addition of position vectors to token embeddings) slightly reduces parameter count and improves training/inference speed, though this is a minor benefit relative to the generalization argument.

How the model learns position without explicit encodings: The attention mask provides relative position information (token A is before token B), but not absolute position (token A is the 47th token). The model learns absolute position implicitly through the cumulative effect of the causal mask — the pattern of which tokens can and cannot attend to each other creates a unique "signature" for each absolute position — and, for document parsing specifically, through the cross-attention to visual features: the visual tokens encode 2D spatial positions, and the decoder learns to attend to visual regions in reading order, which provides an external positional signal aligned with the document layout rather than an arbitrary 1D sequence index.

Training evidence for NoPE effectiveness. The paper states: "We find that the network achieves comparable accuracy to models trained with positional embeddings." No ablation table is provided explicitly comparing Nemotron-Parse with and without positional embeddings, so the reader must take this claim at face value. However, the OmniDocBench reading order metric (0.066 error for the base model, 0.048 for TC — Table 4) suggests that the model does learn meaningful positional structure, since reading order errors would be high if the model could not distinguish token positions.

Multi-token prediction. Section 2.1.2 describes a technique for accelerating inference by predicting $m$ tokens per forward pass rather than one. The architecture modification is minimal:

For predicting $m$ tokens in parallel, the model adds $m-1$ additional linear layers (heads). Let $h_n$ be the final hidden state of the $n$-th token (the last token generated so far). The prediction of the next $m$ tokens proceeds as:

• Token $n+1$ (standard single-token head): Logits are computed as $\text{lhead}(h_n)$, where $\text{lhead}$ is the standard decoder output projection (the "head" layer that maps from hidden dimension to vocabulary dimension). This is the same computation as single-token inference.

• Tokens $n+2$ through $n+m$ (additional multi-token heads): For each subsequent token $n+k$ (where $k \in \{2, ..., m\}$), the logits are computed as:

$\text{lhead}(l_1(h_n + l_2(e_{n+k-1})))$

where $e_{n+k-1}$ is the embedding of the token predicted by the immediately preceding head (the predicted token $n+k-1$), $l_1$ and $l_2$ are learnable linear layers, and $\text{lhead}$ is the shared decoder output projection.

What this equation computes, operationally: For the first additional token ($n+2$), take the hidden state $h_n$ of the last ground-truth token, add a learned transformation of the embedding of token $n+1$ (which was predicted by the standard head), pass this combined representation through another linear layer $l_1$, then through the standard output head $\text{lhead}$. For the next token ($n+3$), repeat this process using the embedding of the predicted token $n+2$ as the input $e_{n+k-1}$. Each additional prediction head reuses the same $h_n$ (the hidden state of the last ground-truth token) and adds only the embedding of its immediate predecessor token.

Why this form — two key design choices:

1. All heads condition on $h_n$ (the last ground-truth token) rather than on intermediate hidden states from previous heads. This is a deliberate simplification. An alternative would be to run the full transformer decoder forward for each additional token — feed each predicted token back through the self-attention and cross-attention layers to produce a new hidden state, then predict the next token. The approach used here avoids the computational cost of running the decoder layers multiple times per generation step. Instead, the linear layers $l_1$ and $l_2$ provide a cheap approximation of "what would the hidden state look like if we had processed the previous predicted token through the full decoder stack?" This makes the multi-token prediction essentially as fast as single-token prediction (plus the cost of $m-1$ small linear layers) while predicting $m$ tokens.

2. The $h_n + l_2(e_{n+k-1})$ addition fuses the full context representation with the preceding token's embedding. The hidden state $h_n$ encodes all context up to the $n$-th token (including cross-attention to the visual features). The linearly transformed embedding $l_2(e_{n+k-1})$ injects information about the most recently predicted token. Adding them together (rather than concatenating or using gating) is computationally cheap and, under the assumption that the previous token's identity is the main additional signal needed to predict the next token (given the full context), may be sufficient.

Training with multi-token prediction. During training, teacher forcing is used for the additional token embeddings: "for token embeddings of additional $n+2..m$ tokens," the model receives the ground-truth embedding $e_{n+k-1}$ (the correct next token) rather than the embedding of the model's own prediction. This is standard practice for sequence models — it prevents error accumulation during training and allows the model to learn each head independently given correct predecessors. At inference time, the model uses its own predicted tokens (greedily, "without token verification"), which means errors in early heads propagate to later heads — a head predicting token $n+3$ receives as input the embedding of the model's prediction for token $n+2$, not the ground truth.

An unexpected benefit. The paper reports a finding that is not obvious from the architecture: "We find that adoption of the multi-token training strategy additionally allows to achieve improved accuracy in the default single-token inference setup, compared to the models trained with a standard protocol." In other words, even when using standard single-token autoregressive decoding at inference time, a model that was trained with the multi-token objective performs better on the task than an identical model trained only with single-token prediction. This is a form of auxiliary task regularization: the multi-token heads force the main hidden state $h_n$ to encode information that is useful for predicting not just the immediate next token but several tokens ahead. This richer representation at each generation step improves the model's overall understanding of the output structure — for example, if the model is about to start a table LaTeX block (\begin{tabular}), the hidden state that predicts \begin must also contain enough information to help the additional heads predict tabular (token $n+2$) and } (token $n+3$), which encourages a more structured internal representation than predicting \begin alone.

Inference speed tradeoffs. The paper does not specify the value of $m$ (the number of simultaneous predictions) used for its reported throughput numbers. If $m=2$, we would expect roughly 2× theoretical speedup; if $m=4$, roughly 4×. In practice, the speedup is less than $m \times$ because the linear heads add computation, the autoregressive dependency between heads means they cannot be parallelized within a generation step beyond the matrix multiplications in $l_1$ and $l_2$, and the main decoder hidden state $h_n$ still needs to be computed (which dominates the per-step cost). The paper reports token-per-second numbers in Table 8 (3800 for base, 4500 for TC) but does not compare against a single-token-only version, so the exact speedup from multi-token prediction cannot be isolated from the token compression in TC.

3.4.3 Prompt Interface and Output Format: One Model, Many Tasks

The prompt interface is the mechanism that allows Nemotron-Parse to be trained on heterogeneous datasets (some providing only plain text, others providing formatted text + bounding boxes + semantic classes) while presenting a unified interface to users at inference time.

Three independent prompt dimensions. The model recognizes prompt tokens across three orthogonal axes:

• Text formatting: <output_markdown>, <output_plain>, or <output_no_text> — controls whether and how text content is formatted.
• Bounding boxes: <predict_bbox> or <no_bbox> — controls whether spatial coordinates are included.
• Semantic classes: <predict_classes> or <no_classes> — controls whether each bounding box receives a class label. This option is used "only together with <bbox>" since classes without spatial coordinates would be meaningless.

The three dimensions are independent, yielding $3 \times 2 \times 2 = 12$ theoretical combinations. The paper excludes "the trivial 'no output' case and any request for classes without boxes," leaving eight valid prompt combinations used during training and inference.

Training with prompt-conditioned data. For each training sample, the prompt is set to match exactly the annotations available in that sample's source dataset. For example:

• A sample from the NVpdftex pipeline, which provides structured markdown/LaTeX text, bounding boxes, and semantic classes, would use the prompt <output_markdown><predict_bbox><predict_classes>.
• A sample from the Common Crawl human-annotated data, which provides only plain text with bounding boxes but no semantic classes, would use <output_plain><predict_bbox><no_classes>.
• A sample from a synthetic OCR dataset that provides only structured text without spatial annotations would use <output_markdown><no_bbox><no_classes>.

This design is crucial: it means the model learns a single conditional distribution $P(\text{output} | \text{image}, \text{prompt})$ that covers all output formats, rather than maintaining separate models or separate output heads for each format. When the model sees <no_bbox> in the prompt, it learns to suppress bounding box generation; when it sees <output_plain>, it learns to produce unformatted text without LaTeX or markdown markup.

Why this works better than separate models: The cross-format training provides implicit regularization and knowledge transfer. When the model learns to produce bounding boxes with semantic classes on richly annotated data (like NVpdftex), it develops internal representations of document layout and semantic structure that improve its plain-text extraction on less annotated data — even though bounding boxes aren't being output, the internal understanding of "this region is a table, that region is a caption" helps the model decide reading order and formatting. Conversely, training on large quantities of plain-text data (where the annotation cost is lower) provides additional OCR signal that improves the model's text recognition accuracy even in structured output modes.

The maximal-information prompt (MIP). At inference time, when maximum capability is desired, the model is used with:

<output_markdown><predict_bbox><predict_classes>

This requests everything the model can produce: formatted text in markdown/LaTeX, bounding box coordinates for every detected text block, and semantic class labels for each block. This is the configuration used for the benchmark evaluations in Tables 2–6.

Output format specification. In MIP configuration, the model interleaves text content, coordinates, and class labels in a specific XML-like format:

<x_0.1152><y_0.2586># NVIDIA Nemotron-Parse 1.1<x_0.8799><y_0.2797> <class_Title>

The format is formally described as:

<x_(\d+)><y_(\d+)>(.*?)<x_(\d+)><y_(\d+)>
<class_([^>]+)>

where the first <x>, <y> pair are the top-left corner coordinates of the bounding box, the text between the coordinate pairs is the text content of that block, the second <x>, <y> pair are the bottom-right corner coordinates, and <class_...> specifies the semantic class.

Coordinate system. Bounding box coordinates are given in relative coordinates "in a scale of 1024 × 1280" (Section 2.2.2). This means coordinates are normalized such that the full page width maps to the range $[0, 1024]$ and the full page height maps to $[0, 1280]$. The aspect ratio (1280/1024 ≈ 1.25) corresponds to a portrait-oriented page with slightly more vertical resolution, which matches typical document aspect ratios (US Letter is 1.294, A4 is approximately 1.414). Using relative coordinates rather than pixel coordinates makes the output format invariant to the input image resolution — the same bounding box specification works whether the input image was rendered at 1648×2048 or 3300×4100 pixels.

Canonical reading order. The bounding boxes are predicted "in a canonical reading order" (Section 2.2.2), which means the sequence of output blocks follows the order a human would read the page. Specifically:

1. Page-Header elements appear first (at the start of the page).
2. Main content elements follow — Text, Section-Header, List-Item, Title, and Formula — "in the order as they would be read by a person looking at the given page."
3. Floating elements appear at the end of the sequence: Footnotes, Page-Footers, Tables, Pictures, and Captions.

The TC variant improves this ordering. Nemotron-Parse-TC "improves upon this canonical reading order, and also includes non-reading-order (floating) elements, i.e., Footnotes, Page-Footers, Tables, Pictures, and Captions within the natural ordering of the page." This means that in the base model, a table in the middle of a page would be output after all body text (at the end), while in the TC variant, it would appear inline at the position where it occurs in the visual layout. This is a significant improvement for downstream consumption — an LLM reading the output sequentially encounters the table at the correct point in the document flow rather than having to piece together body-text-then-tables from separate sections.

The paper does not explain why the TC variant achieves this improved reading order while the base model does not. One plausible reason: the base model's 3200 visual tokens provide very high spatial resolution, which may allow it to overfit to the NVpdftex training data's convention of placing floating elements at the end. The TC variant's coarser 833-token representation may force it to rely more on general layout understanding (which places elements in natural reading order) rather than memorizing a specific output convention. Alternatively, the TC variant may have received different reading-order annotations during training — the paper does not specify whether both variants were trained on identical data.

Semantic class taxonomy. The model recognizes at least 11 semantic classes, inferrable from the reading order description and output examples: Title, Section-Header, Text, List-Item, Formula, Table, Picture, Caption, Footnote, Page-Footer, Page-Header. This taxonomy covers the major structural elements of academic and business documents. The class labels are output as part of the generation stream rather than as a separate classification head, which means the model can (in principle) produce novel class names if trained on data with different class taxonomies — though in practice, the training data's class labels constrain the output vocabulary.

Markdown and LaTeX formatting conventions. The paper specifies several formatting rules for the <output_markdown> prompt:

• Text is formatted as Markdown.
• Formulas and tables are formatted as LaTeX.
• "Inline formulas that do not require any LaTeX syntax to be represented (e.g., consisting only of characters and subscripts/superscripts) remain in markdown format for versatility."

This last point is a practical design choice: simple expressions like x^2 or H_2O do not need LaTeX math delimiters ($x^2$), and outputting them in plain markdown makes the text more readable for downstream consumers that may not render LaTeX. This also explains a quirk noted in Section 4.1: on OmniDocBench, "simple mathematical equations not requiring specialized LaTeX commands" are output in markdown rather than LaTeX math mode, leading to their penalization in the formula evaluation metric — the benchmark expects $x^2$ but the model produces x^2, which is semantically equivalent but syntactically different. This is a deliberate tradeoff favoring output readability over benchmark optimization.

3.4.4 Training Data and the NVpdftex Pipeline

While the model architecture is standard, the paper's primary technical contribution lies in the training data engineering. Section 3.1 and Table 1 detail a training blend that spans millions of examples across synthetic, public, and human-annotated sources.

The NVpdftex pipeline: the core data engine. This is the most significant data contribution and the one the paper describes in most detail. The pipeline is "inspired by Nougat" (Blecher et al., 2024) but addresses a key limitation of the Nougat approach.

What Nougat did: Nougat generated training data by taking arXiv papers in LaTeX format, converting them to HTML using the LaTeXML tool, and then converting the HTML to markdown. This produced document images (from the LaTeX-rendered PDF) paired with markdown text, but the conversion chain (LaTeX → HTML → markdown) introduced two problems: (1) the mapping between rendered page positions and output text was lost during the HTML-to-markdown conversion, so Nougat could not provide bounding boxes; (2) semantic structure (section headers, captions, footnotes) was flattened or lost in the conversion.

What NVpdftex does differently: Instead of an indirect LaTeX → HTML → markdown chain, NVpdftex "couples LaTeX compilation with structured-output extraction in a single pass, preserving tight alignment between the rendered page and the text down to character-level bounding boxes and enabling per-box semantic labels." The key innovation is extending the TeX Live toolchain to "intercept node and character creation, hbox/vbox allocations, token reads, and page output events."

Operationally, this means:

1. A LaTeX document is compiled through a modified TeX engine.
2. As the engine lays out characters, boxes, and paragraphs on the page, the interception mechanism records the exact (x, y) coordinates of every character and the bounding box of every structural element (section header, paragraph, table cell, footnote, etc.).
3. The semantic class of each element is preserved from the LaTeX source — \section{...} produces a Section-Header, \caption{...} produces a Caption, \begin{table}...\end{table} produces a Table.
4. The reading order is determined by the order in which the TeX engine processes the elements, which matches the natural document flow.
5. The final output is a triple: (rendered page image, structured text with markdown/LaTeX formatting, bounding boxes with semantic classes and reading order).

What this enables that Nougat could not:

• Character-level bounding box precision: Training data includes exact spatial coordinates for every text element, enabling the model to learn fine-grained layout understanding.
• Semantic class labels directly from source markup: No need for post-hoc heuristic classification of text blocks — the semantic type is known from the LaTeX source.
• Table structure preservation: Since LaTeX tables have explicit structure (\begin{tabular}, & for columns, \\ for rows), the extracted output can include properly formatted LaTeX table markup rather than flattened cell text.
• Formula formatting preservation: Mathematical equations in LaTeX are preserved in their native format rather than being converted to images or simplified text.

Scale and multilingual augmentation. The NVpdftex pipeline produced 8.3M pages of training data (the "Multilingual arXiv" entry in Table 1). To improve multilingual capabilities, the authors applied machine translation to this dataset in 6 languages (Chinese, German, Spanish, French, Italian, Japanese). Additionally, they applied "LaTeX-level augmentations of fonts, color, and layout to increase the diversity of the datasets" — meaning the same LaTeX source was rendered multiple times with different typographic settings (typefaces, text colors, background colors, column widths, margins) to produce visually diverse training images without changing the ground-truth annotations. This is a form of domain randomization that improves the model's robustness to document appearance variations.

DocLayNet augmentation. DocLayNet (Pfitzmann et al., 2022) is a public dataset of 56K pages with human-annotated bounding boxes and semantic classes (Title, Text, List, Table, Figure). It contains diverse document types beyond academic papers — financial reports, legal documents, manuals, patents, scientific articles, etc. However, DocLayNet's annotations are purely layout-level; they do not include reading order, text formatting, or table/equation structure.

The paper describes augmenting DocLayNet with:

• "autolabeled reading order of the text" — presumably by running a heuristic or a preliminary model to determine which order the text blocks should be read in.
• "text inside images" — OCR applied to figure regions to extract any embedded text.
• "markdown formatting" and "formatting of Table objects and Formulas" — applying formatting to the plain text based on detected structure (e.g., bolding detected headers, converting detected tables to LaTeX).

The resulting data includes multiple output format variants per page: "both plaintext and markdown/LaTeX." This means a single DocLayNet page can appear in training with different prompts — <output_plain> for one variant and <output_markdown> for another — teaching the model to produce both formatted and unformatted output from the same visual input.

Common Crawl human-annotated data. A set of 255K samples from Common Crawl (the web crawl corpus) were annotated by human experts with "plaintext format along with bounding boxes and semantic class labels." This provides real-world diversity that synthetic data cannot capture — web pages contain non-standard layouts, mixed content types, and visual artifacts that clean academic PDFs lack.

The human annotations were further augmented by:

• "autolabeled text inside images" — OCR on embedded figures.
• "structured formatting labels are derived by treating each individual bounding box crop as an individual image and running inference with stage-1 trained Nemotron-Parse" — this is a bootstrapping step: an initial version of Nemotron-Parse was used to generate markdown formatting for the human-annotated bounding boxes.
• "low-quality predictions are filtered out and blanked on the image on the basis of edit distance of the formatting-stripped output to the plaintext labels" — the stage-1 model's formatted output is stripped of markup and compared to the human-annotated plaintext using edit distance; if the edit distance is too high (indicating the model's OCR failed), that bounding box is blanked from the training image to avoid training on bad formatting.
• "page-level formatting, such as multi-level headers, are obtained by running full-page inference and aligning global header formatting with individual text blocks heuristically" — for structural elements that span multiple blocks (like section headers that appear as separate text blocks from their body text), a heuristic alignment process matches the model's full-page formatting predictions back to the individual human-annotated boxes.

Synthetic data components. Several synthetic data sources target specific weaknesses:

• Synthetic tables (26K): Generated in HTML format, converted to LaTeX, and rendered on a page. The paper notes this captures "various layouts, text formatting, sparsity levels, presence of checkboxes, etc." HTML-to-LaTeX conversion ensures the training data includes complex table structures like merged cells, multi-row headers, and nested tables that are common in real documents but underrepresented in academic paper datasets.

• Multilingual dense OCR data (3.5M): "Synthetically generated dense text of multiple languages... rendered on the image. This includes random words, characters, and symbols, in 6 different languages." This is specifically designed to address the observation that "models struggle with dense OCR" — pages where text covers most of the image area with minimal whitespace, such as legal documents, dense tables, or multi-column layouts with small fonts. The random character and symbol generation ensures the model encounters rare glyphs and symbol combinations.

• Multilingual Wikipedia OCR data (9.5M): Wikipedia text in 10 languages (English, French, German, Spanish, Italian, Dutch, Portuguese, Japanese, Korean, Chinese) is "converted to LaTeX formatting and augmented with font, background, and color augmentations." Wikipedia provides diverse subject matter (from science articles to biographies to geographic descriptions) with varied vocabulary, making this a powerful source of domain-diverse text.

Public dataset integration. The paper also incorporates existing table extraction benchmarks into the training data:

• PubTables-1M (585K samples, Smock et al., 2022): Tables from scientific documents with detailed cell-level structure annotations. The paper converts tables "from HTML to LaTeX format" and "autolabeled the remaining elements (if present) in the pages to follow the Nemotron-Parse format" — meaning non-table content on the pages was labeled using the same bootstrapping approach as the Common Crawl data.

• FinTabNet (91.5K samples): Financial tables from corporate annual reports. Same HTML-to-LaTeX conversion and autolabeling process.

• TabRecSet (38.2K samples, Yang et al., 2023): Tables in both English and Chinese, providing multilingual table extraction training data.

• SynthTabNet (480K samples, Nassar et al., 2022): Additional synthetically generated tables with diverse styles.

Why this data blend works: the complementary strengths argument. Each data source compensates for weaknesses in others:

• NVpdftex provides high-quality bounding boxes and semantic classes, but only for academic-style LaTeX documents.
• DocLayNet provides diverse document types (financial, legal, manual) but without text formatting.
• Common Crawl human annotations provide real-world web documents but are expensive to obtain (only 255K samples).
• Synthetic dense OCR provides robustness to edge cases (rare symbols, dense layouts) that real documents undersample.
• Wikipedia provides multilingual coverage and domain diversity at scale (9.5M samples).
• Public table datasets provide specialized table structure training that general document datasets may lack.

A critical observation about data scale: The total training blend is approximately 22M+ samples when summed across all sources (8.3M + 480K + 56K + 255K + 26K + 3.5M + 9.5M + 585K + 91.5K + 38.2K ≈ 22.8M). This is a large but not unprecedented dataset for document parsing — it is smaller than the training sets used by some competing models (which may use hundreds of millions of web-scraped PDF pages) but compensates for scale with quality: the NVpdftex pipeline provides pixel-accurate bounding boxes that web-scraped PDF data cannot match, and the human-annotated Common Crawl data provides ground-truth quality on real-world document types.

Data release. The paper states that "a large portion of the synthetic and human-labeled datasets have been released as part of the Nemotron-VLM-Dataset-V2 release." The NVpdftex generation pipeline is also open-sourced. However, not all data components are released — the specific Common Crawl human-annotated samples may be subject to redistribution restrictions, and the machine-translated NVpdftex data depends on the original arXiv corpus which has its own licensing terms.

3.4.5 Training Procedure and Multi-Token Training Strategy

The paper provides relatively sparse details about the training procedure compared to the extensive data pipeline description. Key training specifications are scattered across Sections 2 and 3, and the paper does not include a dedicated "Training Details" section with hyperparameters, optimizer settings, learning rate schedules, batch sizes, or training duration.

What is specified:

• Multi-token prediction training: For predicting $m$ tokens per step, "we add $m-1 \times 2$ additional linear layers." The multiplication by 2 refers to the two linear layers $l_1$ and $l_2$ in the multi-token head architecture (Section 2.1.2). If $m=3$ (predicting 3 tokens simultaneously), the model has the standard single-token head plus $(3-1) \times 2 = 4$ additional linear layers — one $l_1$ and $l_2$ pair for the second predicted token, and another pair for the third.

• Teacher forcing for multi-token heads: "During training, we use teacher forcing for token embeddings of additional $n+2..m$ tokens." This means the ground-truth token embeddings are fed as input to the $l_2$ layers, not the model's own predictions. This is essential for stable training — if the model's predictions were fed back during training, errors in early heads would compound during backpropagation, creating a noisy gradient signal. Teacher forcing ensures each head is trained independently conditioned on correct predecessor tokens.

• Greedy decoding at inference: "At inference, decoding proceeds greedily without token verification." "Greedy" means selecting the highest-probability token at each step (rather than sampling or beam search). "Without token verification" means the model does not use speculative decoding techniques where a draft model generates candidate multi-token predictions and a verifier model checks them — the multi-token predictions are accepted directly.

• Training on heterogeneous prompt-conditioned data: The model is "trained jointly on heterogeneous datasets that provide different supervision signals" (Section 2.2.1). Each training batch presumably contains samples with different prompt combinations, sampled proportionally from the diverse data sources in Table 1. The paper does not specify the batch composition strategy (e.g., proportional sampling, temperature-based sampling to balance data sources, or curriculum learning where easier formats are trained first).

• Stage-1 and stage-2 training implied by data augmentation: The Common Crawl data augmentation process refers to formatting labels being "derived by treating each individual bounding box crop as an individual image and running inference with stage-1 trained Nemotron-Parse" (Section 3.1, "Common Crawl Data"). This implies at least two training stages: a stage-1 model trained on high-quality data (NVpdftex, public datasets, synthetic data) to achieve reasonable OCR and formatting capability, and a stage-2 model that incorporates the stage-1 model's predictions as autolabels on the human-annotated Common Crawl data. This is a form of self-training or pseudo-labeling, common in modern document parsing pipelines. The paper does not detail the stage-2 training procedure — whether the stage-1 model was fine-tuned on the new data, trained from scratch on the expanded blend, or used as a warm start with different learning rates for original vs. new data.

What is not specified (notable gaps):

• Exact multi-token prediction count $m$: The paper never states how many tokens are predicted simultaneously. This is a significant omission because the throughput numbers depend on this value, and practitioners wanting to reproduce the speed results need to know what $m$ was used.

• Optimizer, learning rate, batch size, training duration: No optimizer type (presumably AdamW, standard for transformer training), no learning rate (or schedule), no batch size, no number of training steps or epochs, no hardware configuration for training. For comparison, a typical VLM training paper would specify these in a "Training Details" appendix or section. Their absence makes the paper less reproducible than it could be.

• Loss function: The paper does not specify the training objective. For an autoregressive language model, the standard loss is teacher-forced cross-entropy: for each position $i$ in the target sequence, minimize the negative log-likelihood of the ground-truth token $t_i^*$ given the visual features and the ground-truth prefix tokens $t_{<i}^*$. The multi-token heads likely use the same cross-entropy loss, summed across all predicted positions. This is standard enough to be assumed, but it should be stated explicitly.

• Training data balancing: With 22M+ samples from diverse sources, how are mini-batches constructed? Are data sources sampled proportionally to their size (which would heavily weight the 9.5M Wikipedia samples and 8.3M NVpdftex samples over the 56K DocLayNet samples)? Or is a temperature-based sampling strategy used to upweight smaller but high-quality sources? This choice significantly affects model behavior — a proportional sampling strategy might underweight the human-annotated Common Crawl data (255K out of 22M ≈ 1.1%), which could explain any weaknesses on in-the-wild web documents.

• Image preprocessing: The paper specifies that the vision encoder processes images at 1648×2048 resolution (for the token count calculation in Section 2.1), but does not state whether this is the training resolution, whether multi-scale training is used, or what preprocessing (normalization, augmentation beyond the LaTeX-level augmentations) is applied.

• Multi-token training's effect on accuracy: The paper claims multi-token training "additionally allows to achieve improved accuracy in the default single-token inference setup" but provides no ablation comparing a model trained with and without multi-token heads. This is a missed opportunity — an ablation table showing single-token, $m=2$, $m=3$, and $m=4$ multi-token training's effect on single-token inference accuracy would strengthen this claim substantially.

Why the training details are sparse. The paper is published as a technical report (arXiv preprint) rather than a full conference paper, which typically allows for less exhaustive method descriptions. Additionally, the emphasis is clearly on the data pipeline and model architecture — the training procedure follows standard practices for vision-language model training, and the authors may view the exact hyperparameters as implementation details rather than novel contributions. However, for a practitioner wanting to reproduce or fine-tune the model, these omissions are significant.

What can be inferred about training from related work. Nemotron-Parse's predecessor, Eclair/Nemoretriever-Parse-1.0 (Karmanov et al., 2025), likely used similar training protocols. If the training setup followed that work, it probably used AdamW with a cosine learning rate schedule, a moderate batch size (128–512), mixed-precision training (bf16), distributed training across multiple GPUs (given the 885M parameter count), and several epochs of training. The reference to "stage-1 trained Nemotron-Parse" suggests a multi-stage training pipeline, with the first stage potentially using higher-quality data and the second stage incorporating pseudo-labeled data. But all of this is inference from context, not from stated parameters.

The overall picture. Nemotron-Parse-1.1's technical approach is best understood as a data-centric engineering contribution wrapped in a standard encoder-decoder architecture. The model's capabilities come from the breadth and quality of the training data — the NVpdftex pipeline provides pixel-accurate annotations that enable spatial understanding; the diverse synthetic and public data sources provide coverage of tables, dense text, and multilingual content; and the prompt interface allows the model to serve multiple output formats from a single set of weights. The architecture choices (NoPE, multi-token prediction, asymmetric vision neck compression, pixel-shuffle for TC) are pragmatic optimizations that improve efficiency and generalization without fundamentally changing the transformer paradigm.

4. Key Insights and Innovations

Innovation 1: Prompt-Conditioned Multi-Format Training as a Unifying Strategy for Heterogeneous OCR Supervision

The most intellectually distinctive contribution of Nemotron-Parse-1.1 is not any single architectural component but rather the training methodology that unifies heterogeneous document annotation sources under a single conditional generation objective. This is a conceptual shift in how to think about training document parsing models, and it addresses a problem that has quietly constrained the field: the expensive fragmentation of annotation types.

What the field did before. Document parsing datasets come in incompatible flavors. Some provide bounding boxes with semantic classes but no text formatting (DocLayNet). Some provide structured markdown/LaTeX text but no spatial annotations (early Nougat-derived datasets). Some provide plain text with spatial coordinates but no semantic labels (human-annotated web data). Some provide specialized table structure annotations but nothing else (PubTables, FinTabNet). The dominant approach has been to either (a) train separate models for each output type — a layout detection model, a text recognition model, a table extraction model — or (b) limit training to a single high-quality data source that provides all annotation types simultaneously, discarding the vast majority of available data. Both are inefficient. The pipeline approach (a) incurs the failure-cascade and latency problems the paper identifies in Section 1. The single-source approach (b) means models are starved of diverse document types simply because those types lack one annotation dimension.

What this paper does differently. The prompt interface described in Section 2.2.1 is deceptively simple — three binary prompt tokens controlling text formatting, bounding boxes, and semantic classes — but its conceptual implication is profound: the model is trained to treat annotation availability as a conditional generation variable, not a data filtering constraint. When a training sample has only plain text and bounding boxes, the prompt is set to <output_plain><predict_bbox><no_classes> and the model learns to generate exactly those outputs. When another sample has formatted text, bounding boxes, and semantic classes, the prompt is <output_markdown><predict_bbox><predict_classes> and the model learns to generate all three. The same model weights handle both cases.

This is not merely a convenience for data engineers. It is a knowledge transfer mechanism: the model learns rich internal representations from richly annotated data (NVpdftex, with pixel-accurate bounding boxes, semantic classes, and LaTeX formatting) and transfers that understanding to sparsely annotated data. When the model processes a Common Crawl page with only plain-text-and-bounding-box annotations, it does so with internal feature representations shaped by having learned semantic classification and markdown formatting on other data sources. The model "knows" about section headers, captions, and footnotes even when it isn't asked to output them — and that knowledge improves its reading-order decisions and text formatting even in plain-text output mode.

Why this is significant beyond raw performance. This framing resolves a tension that has plagued document parsing: the tradeoff between annotation depth (providing all output types for a single page, which is expensive) and annotation breadth (covering many document types, which requires cheaper annotation). By making the output format a conditioned variable, the paper decouples these two dimensions. A training sample contributes whatever annotations it has; the model learns to extract the common visual understanding that underlies all output formats. This is analogous to multi-task learning in NLP (where models trained on multiple tasks develop better representations than single-task models), but applied to the specific challenge of heterogeneous output formats for the same underlying task (document understanding) rather than heterogeneous tasks.

The evidence for this transfer working is indirect but visible in the benchmark results. On the internal reading-order test set (Table 2), Nemotron-Parse achieves 0.958 F1 and 0.109 WER — strong performance despite being trained on data where many samples lack reading-order annotations. The OmniDocBench reading order metric (Table 4) shows the TC variant achieving 0.048 error, competitive with the best pipeline systems that explicitly optimize reading order as a separate stage. These results are consistent with the hypothesis that cross-format training produces better document understanding than format-specific training, because the model cannot rely on format-specific shortcuts and must develop a general internal representation of document structure.

This is an incremental refinement of the multi-task VLM training paradigm rather than a fundamental shift — prompt-conditioned generation is standard in instruction-tuned LLMs — but its application to the specific problem of heterogeneous OCR annotations is novel and practically impactful.

Innovation 2: Token Compression as a Benchmarking and Design Axis, Not an Optimization Afterthought

The paper's treatment of vision token compression — embodied in the Nemotron-Parse-TC variant — elevates compression from an implementation detail to a first-class design axis with systematically evaluated quality-throughput tradeoffs. This is more significant than it sounds because the document parsing field has lacked a systematic characterization of how many visual tokens are "enough."

What the field did before. Visual token counts in document parsing models vary wildly without clear justification. SmolDocling uses 392 tokens. GOT-OCR2.0 uses 256. DeepSeek-OCR-Gundam uses 795. Nougat uses 2352. Nemotron-Parse uses 3201. These numbers reflect architectural accidents and historical precedent more than principled decisions — they are whatever the vision encoder's default patch grid produces for a given input resolution, with whatever ad-hoc pooling the authors happened to add. There is no widely accepted framework for answering the question: "if I reduce vision tokens by 4×, what quality do I lose, and is it worth the speedup?"

What this paper does differently. By releasing two variants — base (3201 tokens) and TC (833 tokens, a 3.8× reduction) — and evaluating both across a comprehensive benchmark suite (Tables 2–7), the paper provides a concrete characterization of the vision-token-quality-throughput surface for document parsing. The results are notably granular:

• Near-identical overall quality: On OmniDocBench (Table 4), the TC variant achieves 0.129 overall error vs. 0.131 for the base model — statistically indistinguishable. On the internal test set (Table 2), TC loses 0.005 F1 (0.958 → 0.953) and gains 0.009 WER (0.109 → 0.111 without masking). These are tiny degradations.
• Task-specific tradeoffs emerge: On OmniDocBench sub-metrics, TC does better on formulas (0.295 vs. 0.288) and tables (0.121 vs. 0.118), but worse on text (0.055 vs. 0.052). The reading order metric improves dramatically (0.048 vs. 0.066) — which the paper attributes to the "vastly improved reading order" of the TC variant that interleaves floating elements inline. On RD-TableBench (Table 6), TC loses only 0.4 points (85.4 vs. 85.8).
• The speed gain is real but modest: 4500 vs. 3800 tokens/second (Table 8), an 18% improvement, which translates to approximately 5 pages/second vs. 4 pages/second.

These results collectively demonstrate that 833 vision tokens are sufficient for competitive document parsing — well below the 3201 tokens used by the base model and far below the 6790 tokens used by InternVL2-76B and MinerU2.0 in Table 4. The TC variant's 833 tokens sit in the same range as DeepSeek-OCR-Gundam (795), which achieves similar overall OmniDocBench performance (0.127 at 795 tokens vs. 0.129 at 833 tokens), suggesting a possible convergence point around 800–1000 tokens as the practical sweet spot for document parsing.

Why this is significant beyond raw performance. This finding provides a decision rule for practitioners that did not previously exist: if you are building a document parsing system and your latency budget is tight, you can compress vision tokens to the ~800-token range and expect negligible quality loss on most metrics, potentially with improved reading order. This is not obvious — a natural prior would be that more tokens = better quality, and compression always hurts. The paper shows this prior is wrong for document parsing at the resolutions and model scales tested.

More importantly, the reading order improvement in the TC variant is a diagnostic finding that raises interesting questions about the relationship between spatial resolution and sequential reasoning. The paper hypothesizes that the base model's high resolution (3201 tokens) may allow it to overfit to the NVpdftex data's convention of placing floating elements at the end, while the TC variant's coarser resolution forces it to rely on general layout understanding that produces natural reading order. If this interpretation is correct, it suggests a broader principle: in vision-to-sequence tasks, excess spatial resolution can enable undesirable memorization of output conventions at the expense of generalization. This is a substantive conceptual insight, not just a speed optimization.

This contribution is incremental in mechanism (pixel-shuffle is a standard operation) but fundamental in its characterization of a design space that the field had not systematically explored. It converts token count from an implementation accident to a deliberate knob with empirically characterized effects.

Innovation 3: The NVpdftex Pipeline — LaTeX-Integrated Structured Extraction as a Data Quality Breakthrough

The NVpdftex data generation pipeline is the engine that makes the rest of the model's capabilities possible, and it represents a genuine methodological advance over prior work in document parsing data generation. While inspired by Nougat, the NVpdftex approach solves a fundamental limitation that constrained all Nougat-derived training datasets.

What Nougat did and why it was limited. Nougat (Blecher et al., 2024) generated training data from arXiv papers by converting LaTeX source to HTML (via LaTeXML) and then to markdown. This produced (document image, markdown text) pairs suitable for training an OCR model to output formatted text. However, the conversion chain (LaTeX → HTML → markdown) had two critical losses:

1. Loss of spatial information: The mapping from rendered page positions to output text was destroyed during the HTML-to-markdown conversion. Nougat could not provide bounding boxes for text elements because the intermediate conversion steps did not preserve character-level layout coordinates.
2. Loss of semantic structure: LaTeX's rich semantic markup (\section, \caption, \footnote, \begin{table}) was flattened during the HTML step into generic formatting tags (<h2>, <div>, etc.) and then into markdown equivalents (##, paragraph breaks). The semantic class of each text element was lost.

These limitations meant that Nougat-trained models could output formatted text from document images, but could not output spatial coordinates or semantic classes — two capabilities that are critical for downstream applications like layout-preserving chunking, document search with spatial highlighting, and structured information extraction.

What NVpdftex does differently. Instead of the indirect LaTeX → HTML → markdown chain, NVpdftex extends the TeX Live compilation engine to intercept internal typesetting events during LaTeX compilation. The key insight is that during compilation, LaTeX knows everything: the exact (x, y) coordinates of every character on the page, the semantic role of every structural element (\section is a section header, \caption is a caption), and the reading order (the order in which the TeX engine processes elements). NVpdftex captures this information directly rather than attempting to recover it from an intermediate representation.

The practical consequence is that NVpdftex produces training data with three annotation types simultaneously — formatted text (markdown/LaTeX), pixel-accurate bounding boxes, and semantic class labels — from a single LaTeX compilation pass. This is a data quality breakthrough because it eliminates the need for post-hoc annotation or heuristic labeling to recover the spatial and semantic information that Nougat's pipeline discarded.

Why this is significant beyond raw performance. The novelty here is not the idea of using LaTeX to generate OCR training data (Nougat did that). It is the architectural insight that intercepting the typesetting engine at compilation time is more powerful than converting the typesetting engine's output through a chain of lossy transformations. This insight has implications beyond the current paper:

• It enables character-level bounding box precision that was previously achievable only through expensive human annotation or heuristic post-processing of rendered output. Character-level precision matters for fine-grained layout tasks like detecting superscript references, aligning footnotes to their references, and distinguishing between adjacent text columns with narrow gutters.
• It preserves semantic labels from the source markup without requiring a separately trained semantic classifier. A \section{Introduction} in LaTeX becomes a Section-Header bounding box in the training data automatically, with zero annotation error. This is significantly more reliable than training a classifier to distinguish section headers from body text based on visual features (font size, boldness, position) — the NVpdftex approach has perfect semantic accuracy by construction.
• It captures reading order directly from the TeX processing sequence, which matches the natural document flow. This is more accurate than heuristic reading-order algorithms (sorting bounding boxes by y-coordinate then x-coordinate, common in pipeline systems) that fail on multi-column layouts, marginalia, and floating elements.

Comparison to prior work. Besides Nougat, the most relevant comparison is to human annotation pipelines for layout datasets like DocLayNet (Pfitzmann et al., 2022). DocLayNet required human annotators to draw bounding boxes and assign semantic classes on 56K pages — a massive annotation effort that nonetheless provides only layout-level annotations (no text content, no reading order). NVpdftex produced 8.3M pages of training data with text + bounding boxes + semantic classes + reading order with essentially zero annotation cost, because the annotation is extracted from the source markup rather than applied post-hoc.

There is an important limitation: NVpdftex only works for LaTeX-source documents. This covers academic papers (arXiv), but not financial reports, legal documents, manuals, or web pages — precisely the document types that DocLayNet and the Common Crawl human-annotated data cover. The paper acknowledges this implicitly by supplementing NVpdftex with exactly those data sources. But for the 8.3M scientific documents in the NVpdftex corpus, the annotation quality is perfect in a way that human annotation can never achieve — no inter-annotator disagreement, no ambiguous edge cases (is this a subsection header or just bold body text?), no spatial imprecision from bounding box drawing.

This contribution is a fundamental methodological advance in OCR training data generation, not an incremental improvement. It moves the data generation paradigm from "render first, annotate after" to "annotate at render time" — capturing structural information at the source rather than attempting to recover it from the rendered output. The open-source release of the pipeline extends this advance to the community.

Innovation 4: No Positional Embeddings as a Deliberate Generalization Strategy for Variable-Length Document Output

The decision to omit positional embeddings from the decoder (Section 2.1.1) is more than a minor architectural tweak — it is a principled design choice motivated by the specific demands of OCR output generation, and it connects Nemotron-Parse to a growing body of evidence that positional embeddings can hurt length generalization in transformers.

What the field did before. Positional embeddings (learned or sinusoidal) are near-universal in transformer language models. The standard reasoning is that self-attention is permutation-invariant — without positional information, the model cannot distinguish "the cat sat on the mat" from "mat the on sat cat the." Learned absolute positional embeddings or sinusoidal encodings are the standard solution. However, this comes with a well-known limitation: the model can only process sequences up to the maximum length seen during training. For sequences longer than this maximum, the positional embeddings for unseen positions must be interpolated or the sequence must be truncated — both of which degrade quality.

For document parsing, this limitation is particularly problematic. A short receipt might produce 200 output tokens; a dense academic page with a complex table and multiple equations might produce 15,000+. A single training budget cannot easily cover this range. Most OCR models either truncate long outputs (losing content from dense pages) or train with artificially short sequences (limiting their utility for complex documents).

What Nemotron-Parse does differently. By omitting positional embeddings entirely, the model's effective maximum output length is limited only by the decoder's self-attention memory (which scales quadratically with sequence length, but modern implementations can handle tens of thousands of tokens). The model can in principle process arbitrarily long sequences if the hardware supports it — there are no learned position vectors to go out of distribution.

The paper justifies this choice with two arguments, one empirical and one architectural:

1. Empirical: "We find that the network achieves comparable accuracy to models trained with positional embeddings" — the NoPE approach doesn't hurt performance on in-distribution sequence lengths.

2. Architectural: In causal decoder-only models, the attention mask itself encodes positional information: token $t_i$ can attend to tokens 1 through $i$ but not $i+1$ onward. This asymmetry breaks permutation invariance without explicit position vectors (Kazemnejad et al., 2023; Zuo et al., 2025). Additionally, the visual tokens from the encoder carry 2D spatial structure, providing an external positional reference aligned with document layout rather than arbitrary 1D sequence order.

The second argument is particularly interesting for OCR. In a standard language modeling task, the model's only source of positional information (without explicit embeddings) is the causal mask. But in OCR, every generated text token is grounded in a specific spatial location on the page via cross-attention to the visual features. The model learns that the 47th output token corresponds to text at position (0.35, 0.12) on the page, while the 48th token corresponds to position (0.36, 0.12). This spatial grounding provides a richer positional signal than any 1D positional embedding could — it is inherently 2D and aligned with document structure.

Why this is significant beyond raw performance. This is a reframing of the positional encoding problem for grounded sequence generation. The standard debate about positional embeddings (learned vs. sinusoidal vs. rotary vs. ALiBi) focuses on how to encode position in the abstract — which functional form best captures 1D sequence order. Nemotron-Parse sidesteps this debate by arguing that for document-grounded text generation, the visual features already carry the necessary positional information, and explicit 1D position signals may even interfere with the 2D spatial structure.

This connects to a broader research direction on length generalization in transformers. Kazemnejad et al. (2023) showed that decoder-only transformers without positional embeddings can learn position implicitly from the causal mask. Zuo et al. (2025) demonstrated that position information emerges in the similarity structure of nearby token embeddings. Nemotron-Parse provides an applied confirmation of these theoretical findings in a production model, and extends them to the cross-modal setting where visual grounding provides an additional positional signal.

Practical implications. For practitioners, this suggests that if your sequence model has access to spatially structured conditioning (images, audio spectrograms, video frames), explicit positional embeddings may be unnecessary or even counterproductive. The grounding signal provides position information that is more semantically meaningful than arbitrary 1D indices. For the specific case of document parsing, this means models can be trained on short documents (to keep training costs manageable) and deployed on long documents without architectural modification — a capability the paper explicitly claims: "allowing inference with significantly longer context lengths."

This contribution is a fundamental conceptual shift for grounded sequence generation, not merely an incremental optimization. It changes the default assumption from "transformers need positional embeddings" to "transformers need positional information — which can come from the attention mask and/or grounded conditioning, not necessarily from explicit embeddings."

Innovation 5: Multi-Token Training as an Auxiliary Objective That Improves Single-Token Quality

The paper's finding that multi-token prediction training improves single-token inference accuracy (Section 2.1.2) is a diagnostic result with implications for how we think about autoregressive model training objectives, not just a throughput optimization.

What the field did before. Multi-token prediction (Gloeckle et al., 2024) is typically framed as a throughput optimization: if you can predict several tokens per forward pass, you reduce the number of sequential decoder steps and therefore reduce latency. The quality implications are usually negative or neutral — the additional prediction heads add parameters and may dilute the training signal for the primary (next-token) objective. The standard expectation is that multi-token prediction trades a small amount of quality for a large amount of speed.

What Nemotron-Parse found. "We find that adoption of the multi-token training strategy additionally allows to achieve improved accuracy in the default single-token inference setup, compared to the models trained with a standard protocol." In other words, even when evaluating in single-token mode (not using the multi-token heads at inference), a model trained with multi-token auxiliary objectives performs better than an identical model trained only on single-token prediction.

How to interpret this. The paper does not provide an ablation quantifying this improvement, but the finding makes mechanistic sense when considered through the lens of representation learning. In standard single-token training, the hidden state $h_n$ at position $n$ is optimized to predict token $n+1$. In multi-token training, $h_n$ is additionally optimized (through the multi-token heads) to predict tokens $n+2$, $n+3$, ..., $n+m$. This forces $h_n$ to encode information that is useful for predicting multiple future tokens, not just the immediate next one.

For structured output tasks like document parsing, this is particularly valuable. Consider a point in the sequence where the model is about to start a table: the hidden state at the \begin token must encode enough information to help predict not just {tabular} (token $n+1$) but also the column specification {lcc} (token $n+2$), the first row's content, and potentially the entire table structure. The multi-token objective encourages the model to develop a more "planful" representation — one that captures the broader structural context needed for coherent table generation rather than just the local syntactic transition.

This is analogous to the well-established finding in NLP that language models trained with auxiliary objectives (e.g., next-sentence prediction in BERT, or future-token prediction in some autoregressive models) develop better representations than models trained only on the primary objective. The auxiliary task acts as a regularizer that discourages the model from learning shortcuts that work for immediate next-token prediction but fail to capture longer-range structure.

Why this is significant beyond raw performance. This finding challenges the standard framing of multi-token prediction as a tradeoff (speed for quality) and suggests it may be better understood as an auxiliary training objective that provides representation learning benefits regardless of whether it is used at inference time. If this result generalizes beyond document parsing to other structured generation tasks (code generation, structured data extraction, formula generation), it would change how practitioners think about training objectives: multi-token prediction should be enabled during training even if single-token decoding is used at inference, because the multi-token objective improves the representations that the single-token decoder depends on.

The paper does not explore this in depth — there is no ablation across different values of $m$, no comparison to other auxiliary objectives, and no analysis of which types of errors the multi-token training reduces. This limits the strength of the claim, but the finding itself is notable and aligns with emerging results in the language modeling literature suggesting that multi-token prediction is more than just a speed optimization.

This contribution is an incremental empirical finding rather than a fundamental theoretical advance, but it is practically significant for practitioners training document parsing models and conceptually significant for how the field thinks about multi-token prediction objectives.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on a collection of public benchmarks and internal test sets rather than a single primary dataset. The main evaluation targets are: (1) an internally curated, human-labeled set of 789 PDF pages drawn from magazines, books, and Common Crawl (Karmanov et al., 2025) for reading-order and OCR quality assessment; (2) the GOT benchmark (Wei et al., 2024) for general OCR metrics; (3) OmniDocBench v1.0 (English subset) for comprehensive document parsing evaluation across text, formulas, tables, and reading order; (4) RD-TableBench (Reducto, 2024) for in-the-wild table extraction; (5) PubTabNet (Smock et al., 2022), FinTabNet (FinTabNet-dataset, 2024), and OmniDocBench for table structure metrics (TEDS/S-TEDS); (6) a multilingual NVpdftex-generated test set of 10,000 dense scientific documents per language across 7 languages for multilingual OCR assessment. The internal 789-page test set uses human annotators "following DocLayNet's labeling scheme, with a key addition of the explicit reading-order annotations" (Section 4.1).

• Base model(s). All experiments use Nemotron-Parse-1.1 (885M parameters: 657M vision encoder + ~228M vision neck/decoder overhead) and its token-compressed variant Nemotron-Parse-1.1-TC (833 vision tokens vs. 3201). The paper does not report results for its predecessor Nemoretriever-Parse-1.0 on most benchmarks, making within-family scaling comparisons impossible from the reported data. The only direct predecessor comparison is implicit — the abstract claims "advances the capabilities of its predecessor" across general OCR, markdown formatting, table parsing, and text extraction from pictures.

• Metrics. The paper uses multiple metrics tailored to different evaluation dimensions. For the internal reading-order test set (Table 2): WER (Word Error Rate, lower is better) and F1 (character-level F1 score, higher is better), with outputs "normalized" and tables/equations/TeX commands excluded from evaluation. For GOT benchmark (Table 3): OCR/F1 Score (character-level F1 for text extraction), Text-Only RO/Edit Distance (normalized edit distance for reading-order text, lower is better), METEOR and BLEU (standard machine translation metrics applied to reading-order evaluation). For OmniDocBench (Table 4): Normalized Edit Distance (lower is better — the paper uses "overall" as the headline metric, which is 1 − accuracy, so lower values indicate fewer errors), reported for overall English performance and sub-categories: text, formula, table extraction, and reading order. For table benchmarks (Tables 5–6): TEDS (Tree Edit Distance-based Similarity, higher is better), S-TEDS (Structure-only TEDS, ignoring cell content), and Table Similarity (percentage metric on RD-TableBench). For multilingual evaluation (Table 7): WER and character-level F1. The OmniDocBench formula metric has a known incompatibility: "since Nemotron-Parse outputs text in markdown format, simple mathematical equations not requiring specialized LaTeX commands... would be represented by markdown text rather than enclosed in LaTeX math environment delimiters, resulting in their penalization" (Section 4.1).

• Baselines. The paper compares against a wide range of systems with varying architectures and scales. Pipeline models: Dolphin (Feng et al., 2025), Marker (Marker, 2025), Mathpix (Mathpix, 2025), MinerU-2.1.1 (Wang et al., 2024), MonkeyOCR-1.2B (Li et al., 2025), PPstructure-v3 (Cui et al., 2025). End-to-end models: Nougat (Blecher et al., 2024), SmolDocling (Nassar et al., 2025), InternVL2-76B (Chen et al., 2024), Qwen2.5-VL-7B and Qwen2.5-VL-72B (Bai et al., 2025), OLMOCR (Poznanski et al., 2025), GOT-OCR2.0 (Wei et al., 2024), OCRFlux-3B (ocr, 2025), GPT4o (OpenAI, 2023), InternVL3-78B (Zhu et al., 2025), dots.ocr (Rednote), Gemini2.5-Pro (AI, 2025), MinerU2.0 (Wang et al., 2024), DeepSeek-OCR series including Tiny, Small, Base, Large, Gundam, and Gundam-M at 200dpi (Wei et al., 2025). For the internal benchmark (Table 2): Kosmos-2.5 in both ocr-mode and md-mode (Lv et al., 2023), GOT in both modes (Wei et al., 2024). For GOT benchmark (Table 3): Pdfium, Docling, Gemini Flash 2.0, Mistral, LandingAI Document Agent, Marker, SmolDocling. For RD-TableBench (Table 6): Reducto, Azure, Textract, Sonnet 3.5, GPT-4o, Llamaparse, Gcloud, Unstructured. For the internal reading-order test set, baselines are evaluated with and without "mask out" (masking headers/footers from images), since GOT "seems to ignore these elements" (Section 4.1). Nemotron-Parse is evaluated without masking.

• Generation budget / compute accounting. The paper does not standardize across models by FLOPs or parameter count — comparisons in Tables 2–7 are purely quality-based. The closest to a compute-normalized comparison is the "Tokens" column in the OmniDocBench table (Table 4), which reports the vision token count for end-to-end models. This provides an implicit compute proxy since cross-attention cost scales with vision token count, but it is incomplete — it omits decoder parameter counts (which vary from ~100M to >70B across compared models), generation token counts (number of output tokens produced per page), and total FLOPs. The TC variant's evaluation (Tables 2–8) implicitly compares two points on the token-efficiency curve (833 vs. 3201 vision tokens) at fixed decoder size, providing a partial compute-quality characterization. Throughput is measured separately in Table 8 as "tokens/second" on a single H100 GPU for bf16 models, computed as "average tokens/seconds end-to-end over 10,000 pages of length 1000 tokens each," yielding roughly 5 pages/second for TC and 4 pages/second for base.

• Cross-validation / statistical protocol. The paper does not report any cross-validation, statistical significance testing, confidence intervals, or error bars on any of the reported metrics. All numbers are point estimates from single evaluation runs. For the internal 789-page benchmark, the evaluation is performed once on the fixed test set — there is no mention of multiple evaluation seeds, bootstrap confidence intervals, or held-out validation folds for hyperparameter selection. The OmniDocBench evaluation uses the standard v1.0 test split (English subset), but the paper does not specify the exact number of test pages used. The multilingual NVpdftex evaluation uses 10,000 documents per language, which is a large enough sample for stable point estimates, but still lacks confidence intervals. For RD-TableBench, Table 6 reports metrics "obtained from public Reducto results" for competing methods, meaning the evaluation protocol may differ slightly between Nemotron-Parse and the baselines (run by different teams, potentially on slightly different data versions). This absence of statistical rigor is common in the document parsing benchmarking literature but makes it difficult to assess whether small differences (e.g., Nemotron-Parse's 0.131 vs. DeepSeek-OCR-Gundam's 0.127 on OmniDocBench) are statistically meaningful.

Main Quantitative Results

OCR and Reading Order on Internal Benchmark

The headline result on the internally curated 789-page test set (Table 2) establishes Nemotron-Parse's strong performance relative to the two end-to-end baselines evaluated: Kosmos-2.5 and GOT-OCR2.0. In the maximal-information prompt (MIP) configuration without masking, Nemotron-Parse achieves 0.109 WER and 0.958 F1, compared to Kosmos-2.5 (ocr-mode) at 0.195 WER / 0.937 F1 and GOT (ocr-mode) at 0.302 WER / 0.818 F1. With masking applied (to account for GOT's header/footer handling limitation), Nemotron-Parse achieves 0.102 WER / 0.957 F1, compared to Kosmos-2.5 at 0.195 WER / 0.937 F1 and GOT at 0.259 WER / 0.879 F1. The TC variant with masking achieves 0.121 WER / 0.949 F1, maintaining a substantial margin over both baselines.

The key comparison is between output modes: Kosmos-2.5 in ocr-mode achieves 0.195 WER while its md-mode achieves 0.249 WER — markdown formatting degrades text accuracy by approximately 28% relative. GOT shows a similar pattern: 0.302 WER (ocr-mode) vs. 0.259 WER (md-mode), where markdown is actually slightly better (0.259 vs. 0.302). Nemotron-Parse's 0.109 WER represents a 44% relative improvement over Kosmos-2.5's best mode and a 64% relative improvement over GOT's best mode. The TC variant's 0.111 WER (unmasked) is nearly identical to the base model's 0.109, suggesting the token compression does not degrade text recognition accuracy on this benchmark.

The paper notes that "for GOT (md), we also mask-out headers and footers from the images, as their model seems to ignore these elements" (Section 4.1). Nemotron-Parse is not evaluated with this masking applied to its own images — the masking comparison (0.102 vs. 0.109 WER) reflects only the removal of headers/footers from the ground-truth reference for metric computation, not from the input images. The paper does not report whether Nemotron-Parse's performance changes when headers/footers are physically masked from input images, which would be a useful robustness check.

OCR on GOT Benchmark

Table 3 places Nemotron-Parse in a broader competitive landscape on the GOT benchmark, evaluating against 7 other systems including commercial offerings (Gemini Flash 2.0, Mistral), open-source tools (Docling, Marker, SmolDocling), and a traditional OCR engine (Pdfium).

The headline numbers: Nemotron-Parse achieves 0.9785 OCR/F1 score and 0.014 text-only reading order edit distance, with METEOR 0.9858 and BLEU 0.9623. The TC variant achieves 0.9755 F1 and 0.014 edit distance, with METEOR 0.9838 and BLEU 0.9582. The strongest competitor is Gemini Flash 2.0 at 0.9915 F1 / 0.0125 edit distance / 0.9934 METEOR / 0.9828 BLEU — a larger, commercial model from Google that outperforms Nemotron-Parse on all four metrics, though the margins are small: 0.013 F1 difference and 0.0015 edit distance difference.

The ranking by F1 score: Gemini Flash 2.0 (0.9915) > Nemotron-Parse (0.9785) > Nemotron-Parse-TC (0.9755) > Mistral (0.9729) > Marker (0.9696) > SmolDocling (0.9588) > LandingAI Document Agent (0.9524) > Docling (0.6744) > Pdfium (0.0036). The Pdfium score of 0.0036 F1 — near-zero — reflects that Pdfium is a PDF text extraction library that produces text directly from the PDF's internal text layer, not an OCR system, and fails completely on the image-based documents in this benchmark.

The reading order edit distance metric reveals more variation: Nemotron-Parse (0.014) and TC (0.014) are second only to Gemini Flash 2.0 (0.0125), with a substantial gap to Mistral (0.0189), Marker (0.0322), and SmolDocling (0.0352). This aligns with the paper's emphasis on reading order as a differentiating capability — Nemotron-Parse is more than 2× better than Marker on reading order edit distance (0.014 vs. 0.0322).

An interesting observation: the TC variant matches the base model exactly on reading order edit distance (0.014 for both) despite the 3.8× token compression, while losing only 0.0030 on F1 and 0.0020 on METEOR. This reinforces the pattern from Table 2: the TC variant preserves reading order quality exceptionally well, consistent with the paper's claim that TC "improves upon this canonical reading order" (Section 2.2.2).

Comprehensive Evaluation on OmniDocBench

Table 4 provides the most detailed multi-dimensional comparison, evaluating Nemotron-Parse against 25 other systems (pipeline and end-to-end) on the English subset of OmniDocBench v1.0. The primary metric is "English overall," reported as normalized edit distance (lower is better — 0.0 is perfect, 1.0 is complete failure).

The headline: Nemotron-Parse achieves 0.131 overall, and Nemotron-Parse-TC achieves 0.129 — the TC variant slightly outperforms the base model on aggregate. This places both variants in the upper tier of end-to-end models. The ranking among end-to-end models (excluding pipeline systems at the top): DeepSeek-OCR-Gundam-M†200dpi (0.123) > DeepSeek-OCR-Gundam (0.127) > Nemotron-Parse-TC (0.129) > Nemotron-Parse (0.131) > MinerU2.0 (0.133) > DeepSeek-OCR-Base (0.137) > DeepSeek-OCR-Large (0.138) > Gemini2.5-Pro (0.148) > PPstructure-v3 (0.152) > MonkeyOCR-1.2B (0.154) > MinerU-2.1.1 (0.162) > dots.ocr (0.182) > Mathpix (0.191) > Qwen2.5-VL-72B (0.214) > InternVL3-78B (0.218) > DeepSeek-OCR-Small (0.221) > GPT4o (0.233) > OCRFlux-3B (0.238) > GOT-OCR2.0 (0.287) > Marker (0.296) > Qwen2.5-VL-7B (0.316) > OLMOCR (0.326) > Dolphin (0.356) > InternVL2-76B (0.44) > Nougat (0.452) > SmolDocling (0.493) > DeepSeek-OCR-Tiny (0.386 — actually placed between InternVL2-76B and Nougat in the table, but listed here for completeness — the exact ordering in Table 4 puts Tiny at 0.386, between InternVL2-76B at 0.44 and Nougat at 0.452, which appears to be an inverted ordering since 0.386 < 0.44; this likely reflects the table being sorted by a metric other than overall score, or an error in table formatting).

Pipeline models occupy the top positions: Dolphin (0.356), Marker (0.296), Mathpix (0.191), MinerU-2.1.1 (0.162), MonkeyOCR-1.2B (0.154), PPstructure-v3 (0.152). Note: Table 4 reports these numbers as "English overall" with the same metric (normalized edit distance), but the pipeline models' numerical values (0.356 for Dolphin, 0.296 for Marker) are higher (worse) than end-to-end models like Nemotron-Parse (0.131). This appears paradoxical given the paper's framing that pipeline systems achieve higher accuracy. The resolution is that OmniDocBench's overall metric likely aggregates across all sub-categories (text, formulas, tables, reading order), and pipeline models that excel at one sub-category may be penalized on others — Dolphin achieves 0.352 on text and 0.258 on tables (both strong) but 0.465 on formulas and 0.35 on reading order, dragging the aggregate up. In contrast, end-to-end models achieve more balanced (but sometimes lower peak) performance.

The sub-category breakdown reveals Nemotron-Parse's strengths and weaknesses:

• Text extraction: Nemotron-Parse achieves 0.052, TC achieves 0.055. Both are competitive: Dolphin achieves 0.352, Marker 0.085, Nougat 0.365, SmolDocling 0.262. The best end-to-end performer is MinerU2.0 at 0.045, with dots.ocr†200dpi at 0.032. Nemotron-Parse's 0.052 is in the top tier of end-to-end models but behind the best pipeline systems (Dolphin 0.352 is substantially worse — higher error — indicating Nemotron-Parse significantly outperforms Dolphin on text).
• Formula extraction: Nemotron-Parse achieves 0.288, TC achieves 0.295. This is mid-tier: Dolphin achieves 0.465, Nougat 0.488, SmolDocling 0.753 (significantly worse), while InternVL2-76B achieves 0.543, Qwen2.5-VL-7B 0.376. The paper explicitly notes the markdown-vs-LaTeX formatting incompatibility that penalizes Nemotron-Parse on this metric: simple equations output in markdown rather than LaTeX math delimiters are counted as errors. The true formula extraction quality may be higher than these numbers indicate.
• Table extraction: Nemotron-Parse achieves 0.118, TC achieves 0.121. This is the model's strongest sub-category relative to competitors: Dolphin achieves 0.258, Nougat 0.572, SmolDocling 0.729, Qwen2.5-VL-7B 0.598. The TC variant's 0.121 is slightly better than the best DeepSeek-OCR variants (Gundam-M†200dpi at 0.147, Gundam at 0.134) and competitive with the best pipeline systems (MinerU2.0 at 0.15, PPstructure-v3 at 0.162, MonkeyOCR at 0.164). Only Dolphin (0.258) is in the same range among pipeline systems.
• Reading order: Nemotron-Parse achieves 0.066, TC achieves 0.048. The TC variant's reading order score of 0.048 is among the best across all systems: better than Dolphin (0.35), Marker (0.116), Mathpix (0.108), Nougat (0.382), SmolDocling (0.227), and most DeepSeek-OCR variants (0.056–0.283). Only dots.ocr†200dpi (0.04) and MinerU2.0 (0.066) are competitive. The TC variant's 0.048 vs. the base model's 0.066 represents a 27% relative improvement, which the paper attributes to the improved reading order that includes floating elements inline.

The "Tokens" column in Table 4 provides the compute-efficiency context. Nemotron-Parse uses 3201 tokens, which is substantially higher than most end-to-end competitors: DeepSeek-OCR-Tiny (64), DeepSeek-OCR-Small (100), DeepSeek-OCR-Base (256), GOT-OCR2.0 (256), SmolDocling (392), DeepSeek-OCR-Large (400), DeepSeek-OCR-Gundam (795). However, the TC variant at 833 tokens is directly comparable to Gundam (795) and achieves nearly identical overall performance (0.129 vs. 0.127). Models using more tokens (Nougat at 2352, Qwen2.5-VL-72B at 3949, InternVL3-78B at 6790) do not necessarily achieve better results — InternVL3-78B at 6790 tokens achieves 0.218 overall, worse than Nemotron-Parse-TC at 833 tokens (0.129). This is a strong empirical demonstration that vision token count beyond ~800–1000 provides diminishing returns for document parsing, and that model architecture and training data matter more than raw visual resolution.

Table Extraction on Specialized Benchmarks

Tables 5 and 6 evaluate table extraction capability specifically, complementing the OmniDocBench table sub-metric.

Table 5 reports TEDS and S-TEDS on three table benchmarks. On RD-TableBench: Nemotron-Parse achieves 86.2 TEDS / 79.9 S-TEDS, and TC achieves 85.3 / 79.6 — a loss of approximately 1 point on TEDS and negligible change on S-TEDS. On PubTabNet: Nemotron-Parse achieves 81.3 TEDS / 93.99 S-TEDS, and TC achieves 80.9 / 93.6 — again minimal degradation. On OmniDocBench 1.0 English: Nemotron-Parse achieves 82.68 TEDS / 89.06 S-TEDS, and TC achieves 84.73 / 91.44 — notably, TC outperforms the base model on OmniDocBench table metrics, consistent with the reading order improvement pattern.

Table 6 compares on RD-TableBench against 8 external systems. Nemotron-Parse achieves 85.8 Table Similarity, TC achieves 85.4. The only system outperforming Nemotron-Parse is Reducto at 90.2 — a 4.4-point margin. The remaining competitors trail: Azure (82.7), Textract (80.9), Sonnet 3.5 (80.7), GPT-4o (76.0), Llamaparse (74.6), Gcloud (64.6), Unstructured (60.2). The gap from Nemotron-Parse (85.8) to the third-place Azure (82.7) is 3.1 points, suggesting a clear tier boundary between Reducto, Nemotron-Parse, and the rest. The TC variant's 85.4 loses only 0.4 points relative to the base model, preserving the second-place position.

Multilingual OCR

Table 7 reports plaintext WER and F1 on the NVpdftex multilingual test set (10,000 dense scientific documents per language with font and color augmentations). The results show remarkably consistent performance across 7 languages:

• English: 0.03 WER, 0.98 F1
• German: 0.06 WER, 0.96 F1
• French: 0.05 WER, 0.97 F1
• Italian: 0.05 WER, 0.97 F1
• Spanish: 0.04 WER, 0.97 F1
• Chinese: 0.03 WER, 0.98 F1
• Japanese: 0.03 WER, 0.98 F1

The F1 scores are all ≥0.96, with English, Chinese, and Japanese at 0.98. The WER ranges from 0.03 (English, Chinese, Japanese) to 0.06 (German) — a 2× difference but still very low in absolute terms. The paper does not report the TC variant's multilingual performance, which is a notable gap since token compression might affect non-Latin scripts (Chinese, Japanese) differently than Latin scripts.

The caveat stated in Section 4.3 is important: "for Chinese, Japanese, and Korean we find Nemotron-Parse to perform well in the scientific domain as well as standard pdf documents, with limited support for in-the-wild images/documents in these languages." The 10,000-document NVpdftex test set is scientific domain only, so these numbers reflect performance on clean, rendered LaTeX documents — not photographs, scanned documents, or web pages. The multilingual claims should be understood as applying primarily to the digital-born document domain.

Ablation Studies and Robustness Checks

Vision token compression (Base vs. TC across all benchmarks): The TC variant serves as the primary ablation, testing whether 833 vision tokens are sufficient relative to 3201. Across Tables 2–7, the TC variant consistently matches or nearly matches the base model with only two notable exceptions where it outperforms: OmniDocBench reading order (0.048 vs. 0.066, a 27% improvement) and OmniDocBench table metrics (84.73/91.44 TEDS/S-TEDS vs. 82.68/89.06). The consistent pattern — near-identical text quality, improved reading order, slightly better tables — suggests the token compression forces the model to learn more robust layout representations that generalize better to the diverse OmniDocBench documents. This is not a standard ablation (it's a separate model variant), but it functions as one by testing the sensitivity of quality to the primary architectural hyperparameter (vision token count).

Header/footer masking on internal benchmark (Table 2, "mask out" column): For the competing baselines (Kosmos-2.5, GOT), masking headers and footers from evaluation changes results minimally for Kosmos-2.5 (WER: 0.195 masked vs. 0.195 unmasked; F1: 0.937 vs. 0.937) but substantially for GOT in markdown mode (WER: 0.259 masked vs. 0.249 unmasked; F1: 0.879 vs. 0.890). Nemotron-Parse's own masked vs. unmasked comparison shows negligible change: WER 0.102 vs. 0.109, F1 0.957 vs. 0.958. This confirms that Nemotron-Parse successfully extracts headers and footers (they are present in the output, so masking them from evaluation changes the score minimally), unlike GOT which "seems to ignore these elements."

Output format mode on internal benchmark (Table 2, ocr-mode vs. md-mode for baselines): For Kosmos-2.5, markdown mode degrades WER from 0.195 to 0.249 (27% relative increase) and F1 from 0.937 to 0.890. For GOT, markdown mode improves WER from 0.302 to 0.259 (14% relative decrease) with a mixed effect on F1 (0.818 vs. 0.879 masked, but 0.818 vs. 0.890 unmasked — the masking interaction complicates interpretation). Nemotron-Parse is evaluated only in its maximal-information prompt (which includes markdown formatting), achieving better scores than either baseline in either mode — a demonstration that Nemotron-Parse's single MIP configuration outperforms specialized single-mode configurations from competitors.

Multi-token training benefit (Section 2.1.2, no dedicated ablation table): The paper claims that "adoption of the multi-token training strategy additionally allows to achieve improved accuracy in the default single-token inference setup" but provides no ablation comparing single-token-trained and multi-token-trained models on any benchmark. This claim is stated as an observation rather than experimentally validated in the paper. The actual $m$ value used for multi-token training is not reported, nor is the magnitude of the claimed improvement. This is a significant gap — the claim is interesting but unsubstantiated in the paper as written.

Reading order canonical vs. inline (Section 2.2.2, implicit ablation via TC comparison): The base model places floating elements (Footnotes, Page-Footers, Tables, Pictures, Captions) at the end of the output, while the TC variant interleaves them inline. The OmniDocBench reading order metric shows TC (0.048) substantially better than base (0.066), providing an implicit ablation of output ordering conventions. However, this is confounded with the token compression — it's unclear whether the reading order improvement comes from the different output convention, the coarser visual representation, or both. A clean ablation would test the base model with inline ordering or the TC variant with end-of-page ordering.

No ablation of model scale or architecture variants: The paper does not report results for smaller decoder sizes (fewer layers), larger vision encoders, different compression ratios between 833 and 3200 tokens, or alternative neck architectures. There is no comparison to training the same data with a different vision backbone (e.g., a ViT-L instead of ViT-H). The design space exploration is limited to exactly two points (base and TC), which characterizes the vision-token-quality tradeoff but does not map the full surface.

No ablation of data components: The paper does not report which training data sources contribute most to performance (e.g., removing synthetic dense OCR, removing NVpdftex, removing Common Crawl). Given the emphasis on data diversity as the core enabler (Section 3.1), ablations showing the marginal contribution of each data source would substantially strengthen the paper's claims about data importance. Without such ablations, the reader cannot assess whether the 22M+ training samples could be reduced to, say, 10M with minimal quality loss, or whether specific data sources are critical for specific capabilities.

No ablation of prompt conditioning: The paper does not compare prompt-conditioned training (the multi-format approach) against training separate models for each output format. The claim that prompt conditioning improves knowledge transfer across formats is plausible but untested — an ablation comparing a multi-format model against single-format models of equal total capacity would be needed to validate this.

Critical Assessment

The paper's central claim is that Nemotron-Parse-1.1 is a "lightweight document parsing and OCR model" that "delivers improved capabilities across general OCR, markdown formatting, structured table parsing, and text extraction from pictures, charts, and diagrams" while achieving "competitive accuracy on public benchmarks" (Abstract). The experiments support specific aspects of this claim to varying degrees.

Does the paper demonstrate that Nemotron-Parse-1.1 "advances the capabilities of its predecessor"? This claim is not directly tested by any experiment in the paper. No benchmark table compares Nemotron-Parse-1.1 against Nemoretriever-Parse-1.0. The reader must accept the advancement claim on faith or infer it from the predecessor's published results (Karmanov et al., 2025). For a paper whose abstract leads with "advances the capabilities of its predecessor," the absence of a direct comparison is a significant experimental gap. The predecessor's benchmark results are not reproduced in this paper, making it impossible to quantify the magnitude of improvement from the current paper alone.

Does the paper demonstrate competitive accuracy on public benchmarks? Yes, with qualifications. On the GOT benchmark (Table 3), Nemotron-Parse ranks second behind Gemini Flash 2.0 on F1 score (0.9785 vs. 0.9915) and second on reading order edit distance (0.014 vs. 0.0125) — clearly competitive. On OmniDocBench (Table 4), Nemotron-Parse (0.131 overall) ranks behind several DeepSeek-OCR variants among end-to-end models (e.g., Gundam-M at 0.123, Gundam at 0.127) and behind pipeline systems. It is competitive but not leading. On RD-TableBench (Table 6), Nemotron-Parse ranks second (85.8) behind Reducto (90.2) with a clear gap to third place (Azure 82.7) — strongly competitive. The TC variant's OmniDocBench results (0.129 overall, 0.048 reading order) are the most impressive: at 833 vision tokens, it matches or exceeds models using 2–8× more tokens. The "competitive" claim is well-supported across multiple benchmarks, but "state-of-the-art" would be an overstatement — the model consistently places in the top tier without leading any single benchmark.

Does the paper demonstrate that the model is "lightweight" in a meaningful sense? At 885M parameters with a 256M-parameter decoder, Nemotron-Parse-1.1 is compact compared to general-purpose VLMs like Qwen2.5-VL-72B (72B parameters) or InternVL3-78B (78B), and it significantly outperforms both on OmniDocBench (0.131 vs. 0.214 and 0.218 respectively). However, it is not the smallest competitive model — DeepSeek-OCR-Tiny uses 64 vision tokens (vs. 3201 for Nemotron-Parse base) and achieves 0.386 overall on OmniDocBench, trading significant quality for extreme compactness. SmolDocling uses 392 vision tokens and achieves 0.493 overall. The most relevant comparison is against models in the 800–4000 vision token range, where Nemotron-Parse is strong but not uniquely efficient — DeepSeek-OCR-Gundam at 795 tokens achieves 0.127 vs. Nemotron-Parse-TC at 833 tokens achieving 0.129, making them essentially tied. The "lightweight" claim holds when compared to 70B+ generalist VLMs, but is less distinctive when compared to similarly-sized document-specialized models. The paper would be strengthened by a parameter-count-per-benchmark-point comparison or a FLOPs-normalized evaluation that accounts for both vision token count and decoder size.

Does the paper demonstrate that the TC variant offers "a 20% speed improvement with minimal quality degradation"? Yes, with specific numbers. Table 8 reports 4500 tokens/second for TC vs. 3800 for base — an 18.4% improvement (the paper rounds to 20%). Quality degradation is characterized across Tables 2–7: TC loses 0.005 F1 on the internal benchmark (Table 2), 0.003 OCR/F1 on GOT (Table 3), gains 0.002 on OmniDocBench overall (Table 4), loses 0.4 points on RD-TableBench similarity (Table 6), and loses 0.4–1.0 points on TEDS metrics (Table 5). These are genuinely minimal degradations, and the TC variant sometimes outperforms the base model (OmniDocBench overall, reading order, table metrics). The 20% speed claim is numerically supported (18.4%, close to the stated 20%). The "minimal quality degradation" claim is strongly supported — in fact, the degradation is arguably negligible or even negative (TC is sometimes better).

Does the paper demonstrate the effectiveness of the prompt-conditioned multi-format training? No. This is a major architectural claim — that training on heterogeneous annotations with a conditional prompt interface enables knowledge transfer and versatility — but no experiment isolates its effect. There is no comparison of prompt-conditioned training vs. single-format training, no ablation of specific prompt combinations, and no analysis of whether the model actually transfers knowledge across formats (e.g., whether training with bounding box data improves plain-text extraction). The model's strong performance across formats is consistent with the claim but does not demonstrate it causally. An experiment comparing a prompt-conditioned model against an ensemble of format-specific models of equal total capacity would be needed to support this claim.

Does the paper demonstrate the effectiveness of NoPE (no positional embeddings)? Not experimentally. The paper states "We find that the network achieves comparable accuracy to models trained with positional embeddings" (Section 2.1.1) but provides no ablation comparing models with and without positional embeddings on any benchmark. The OmniDocBench reading order score (0.066/0.048) is consistent with the model having learned positional structure, but it does not demonstrate that removing positional embeddings was beneficial or even neutral. The claim is purely architectural/philosophical without experimental backing in this paper.

What experiments would strengthen the paper?

1. Direct predecessor comparison: Evaluating Nemoretriever-Parse-1.0 on the same benchmarks (OmniDocBench, GOT, internal test set) would quantify the claimed advancement.
2. Data ablation studies: Removing individual data sources (NVpdftex, Common Crawl, synthetic dense OCR, public table datasets) and measuring the impact on each OmniDocBench sub-metric would identify which data sources drive which capabilities.
3. Multi-token training ablation: Comparing models trained with m=1, m=2, m=3, m=4 multi-token prediction heads on both single-token and multi-token inference accuracy would validate the claimed accuracy benefit and characterize the speed-quality tradeoff curve.
4. Positional embedding ablation: Training otherwise identical models with and without positional embeddings and comparing OmniDocBench performance would validate the NoPE claim.
5. Prompt conditioning ablation: Training single-format models (e.g., plain-text-only, markdown-only) of equal total capacity and comparing against the prompt-conditioned multi-format model would validate the knowledge transfer claim.
6. Intermediate compression ratios: Evaluating models with vision token counts between 833 and 3200 (e.g., 1600, 2400) would map the full vision-resolution-quality curve rather than just two points.
7. Statistical significance: Reporting confidence intervals or multiple evaluation seeds on the 789-page internal benchmark would clarify whether the small differences between Nemotron-Parse and competitors (e.g., 0.131 vs. 0.133 vs. 0.137 on OmniDocBench) are statistically meaningful.
8. Latency, not just throughput: The 20% speed improvement is reported in tokens/second (throughput), but interactive applications care about time-to-first-token and total latency per page. A latency comparison between base and TC would complement the throughput numbers.
9. FLOPs-normalized comparison: Normalizing OmniDocBench scores by total parameters or inference FLOPs (incorporating both vision tokens and decoder size) would strengthen the "lightweight" claim with a quantitative efficiency metric.
10. In-the-wild evaluation for multilingual claims: The multilingual evaluation (Table 7) uses clean NVpdftex-generated scientific documents. Evaluating on real-world multilingual documents (scanned pages, photographs, web pages) would test the stated limitation that CJK support is "limited" for in-the-wild documents.

Are there genuine experimental weaknesses that undermine specific claims?

• The predecessor advancement claim is entirely untested. A reader evaluating Nemotron-Parse-1.1 as a standalone paper has no experimental basis for assessing whether it improves upon Nemoretriever-Parse-1.0. This is the most significant disconnect between the paper's framing and its experimental content.
• The multi-token accuracy benefit claim is stated without evidence. This is a peer-review-level weakness — a claimed finding (Section 2.1.2) that would be interesting if true, but is presented without the supporting experiment that would normally be required to make such a claim.
• The OmniDocBench formula metric is known to be unfair to Nemotron-Parse (due to the markdown-vs-LaTeX formatting convention), yet the paper does not provide a corrected formula metric or evaluate with an alternative benchmark that accepts markdown-formatted equations. This makes the formula sub-score (0.288/0.295) difficult to interpret — the true formula extraction quality is unknown.
• The vision token count comparison in Table 4 is incomplete as a compute-efficiency metric. Token count affects cross-attention cost but ignores decoder size (which varies from ~100M to 78B parameters across compared models) and output length (which varies by page). A model with 64 vision tokens and a 7B-parameter decoder (DeepSeek-OCR-Small, 0.221 overall) may have higher total inference FLOPs than Nemotron-Parse (3201 tokens, 256M decoder, 0.131 overall) despite the token count ratio. The paper does not account for this.
• The TC variant's reading order improvement is confounded with token compression. The paper attributes improved reading order to the TC variant's different output convention (inline floating elements), but cannot rule out that the compression itself (reduced overfitting to training data conventions) or differences in TC-specific training data are responsible. A clean ablation is absent.
• The test set sizes are not reported for all benchmarks. The internal benchmark is 789 pages (stated). The GOT benchmark size is not stated. The OmniDocBench English subset size is not stated. The RD-TableBench size is not stated. For interpreting whether point estimates are reliable, test set size matters — a 0.002 difference on OmniDocBench could be meaningful on 10,000 pages or noise on 100 pages.

Overall, the experimental evaluation demonstrates that Nemotron-Parse-1.1 is a strong, versatile, and efficient document parsing model that competes effectively with both larger end-to-end systems and multi-stage pipelines across a broad range of benchmarks. The TC variant's near-lossless compression is the most robustly validated finding — the consistency of this result across four independent benchmarks (Tables 2, 3, 4, 6) and four metrics (F1, WER, edit distance, TEDS) provides strong evidence that 833 vision tokens are sufficient for document parsing at this model scale. However, the paper's specific claims about its training methodology (prompt conditioning, multi-token training benefit, NoPE effectiveness) are stated as design choices rather than experimentally validated, and the headline claim of advancing beyond the predecessor is untested in the reported experiments. The paper is best understood as a strong system demonstration and empirical characterization rather than a hypothesis-testing research contribution — it shows what the model can do, and characterizes the vision-token-quality tradeoff thoroughly, but does not causally establish why the model achieves its performance.

6. Limitations and Trade-offs

6.1 The Predecessor Advancement Claim Is Empirically Untested

The assumption or constraint. The paper's abstract and introduction lead with the claim that Nemotron-Parse-1.1 "advances the capabilities of its predecessor, Nemoretriever-Parse-1.0" across "general OCR, markdown formatting, structured table parsing, and text extraction from pictures, charts, and diagrams." This is the paper's primary framing — a successor model delivering meaningful improvements. However, no experiment in the paper directly compares Nemotron-Parse-1.1 against Nemoretriever-Parse-1.0 on any benchmark. The predecessor's results are not reproduced in any table, and no side-by-side evaluation on OmniDocBench, GOT, RD-TableBench, or the internal test set is provided.

The consequence. A practitioner evaluating whether to upgrade from Nemoretriever-Parse-1.0 to Nemotron-Parse-1.1 has no quantitative basis for assessing the magnitude of improvement. The advance could be substantial (e.g., 20% relative improvement on table extraction) or marginal (e.g., 2% on most metrics), and the paper provides no way to distinguish these scenarios. For a paper whose stated contribution is incremental improvement over a known predecessor, the absence of a direct comparison fundamentally weakens the value proposition — the reader cannot determine whether the upgrade is worth the deployment effort without running independent evaluations. This is the most striking disconnect between the paper's framing and its experimental content.

What evidence exists in the paper. None. No table, figure, or section compares the two models quantitatively. The predecessor is cited in the abstract and Section 1 as the baseline being improved upon, but it never appears in the experimental results. The internal 789-page test set (Table 2) is "drawn from magazines, books, and the Common Crawl corpus" and was previously used to evaluate the predecessor in Karmanov et al. (2025), but the predecessor's scores on this benchmark are not reproduced in the current paper. The practitioner must locate the predecessor's original paper to find comparable numbers, and even then, differences in evaluation protocol (masking, normalization, metric computation) make cross-paper comparisons unreliable.

Mitigation status. The paper does not acknowledge this gap. The predecessor is mentioned as the baseline being advanced, but no explanation is offered for why direct comparisons are absent. The open-source release of both models (the predecessor was also released) enables practitioners to run their own evaluations, but this places the evaluation burden on the user rather than the paper providing the evidence to support its central framing claim. This is a significant transparency gap that should be addressed either by releasing a direct comparison table or by tempering the advancement claim language.

6.2 The Difficulty Estimation Cost for the Training Data Pipeline Is Not Quantified

The assumption or constraint. The paper's core training data engine — the NVpdftex pipeline — generates high-quality annotations by intercepting LaTeX compilation events to capture character-level bounding boxes, semantic classes, and reading order simultaneously. This requires the source document to exist in LaTeX format with correct, compilable markup (Section 3.1). The data generation pipeline is described as coupling "LaTeX compilation with structured-output extraction in a single pass" and is reported to have produced 8.3M pages of training data (Table 1, "Multilingual arXiv").

The consequence. LaTeX-source documents are a specific, constrained subset of all real-world documents. Academic papers on arXiv are available in LaTeX, but the vast majority of documents encountered in production deployments — scanned legal contracts, photographed receipts, web pages, financial reports, historical archives — have no LaTeX source. The NVpdftex pipeline produces zero training data for these document types. This means the model's training on high-quality annotations (pixel-accurate bounding boxes, ground-truth semantic classes, perfect reading order) is concentrated entirely on academic-style documents. The paper supplements this with other data sources (DocLayNet, Common Crawl human annotations, synthetic data) to cover other document types (Section 3.1), but those supplementary sources lack the annotation quality of NVpdftex — DocLayNet provides layout boxes without text or reading order, Common Crawl provides human-annotated plaintext with bounding boxes, and synthetic tables are generated in HTML rather than extracted from real financial documents. The consequence is a training data quality skew: the model's strongest supervision signal (NVpdftex) covers only a narrow document type, while the document types that dominate production use cases receive weaker supervision.

A practitioner deploying Nemotron-Parse for, say, parsing legal contracts or medical forms should expect lower quality than the benchmark numbers suggest, because those document types are underrepresented in the high-quality NVpdftex training data and are covered only by the 56K DocLayNet samples and 255K Common Crawl human-annotated samples — together less than 1.5% of the total training blend by sample count. The paper does not provide per-document-type breakdowns of benchmark performance, so the magnitude of this skew cannot be assessed from the reported results.

What evidence exists in the paper. Indirect evidence only. The training data composition in Table 1 shows that NVpdftex (8.3M samples) dominates by volume, followed by multilingual Wikipedia OCR (9.5M, also synthetic). Real-world human-annotated data (Common Crawl, 255K) and diverse-layout human-annotated data (DocLayNet, 56K) are a small fraction. The paper acknowledges the document type limitation implicitly when it states that for Chinese, Japanese, and Korean, the model performs well "in the scientific domain as well as standard pdf documents, with limited support for in-the-wild images/documents" (Section 4.3). This acknowledgment is limited to multilingual CJK, but the same logic applies to English-language non-scientific documents. The OmniDocBench English subset (Table 4) includes diverse document types beyond academic papers, and Nemotron-Parse achieves competitive performance there (0.131 overall), suggesting the supplementary data sources partly compensate — but OmniDocBench's document type distribution is not reported in the paper, so it is unclear how representative it is of real-world document diversity.

Mitigation status. The paper partially addresses this by incorporating non-LaTeX data sources (DocLayNet, Common Crawl, synthetic tables) and by open-sourcing the NVpdftex pipeline so other practitioners can generate training data from their own LaTeX corpora. However, the fundamental constraint — that pixel-accurate annotations require LaTeX source — is inherent to the approach and is not mitigated. The paper does not propose or evaluate methods for generating comparable annotation quality from non-LaTeX documents (e.g., iterative model-in-the-loop annotation, weakly supervised layout detection). The limitation is inherent to the data generation methodology, not an oversight, but the paper's emphasis on the NVpdftex pipeline's quality without equally quantifying its domain restriction gives an incomplete picture of the training data's coverage.

6.3 Multi-Token Training and NoPE Claims Are Stated Without Supporting Experiments

The assumption or constraint. The paper makes two specific methodological claims that, if true, would be practically significant for practitioners building document parsing models:

1. Multi-token training improves single-token quality (Section 2.1.2): "We find that adoption of the multi-token training strategy additionally allows to achieve improved accuracy in the default single-token inference setup, compared to the models trained with a standard protocol."
2. NoPE (no positional embeddings) achieves comparable accuracy (Section 2.1.1): "We find that the network achieves comparable accuracy to models trained with positional embeddings, while allowing inference with significantly longer context lengths."

These claims are presented as empirical findings that motivated design decisions. However, neither claim is supported by any experiment, ablation table, or comparative evaluation in the paper. There is no comparison of models trained with and without multi-token heads on any benchmark. There is no comparison of models trained with and without positional embeddings. The actual multi-token prediction count m is never stated.

The consequence. A practitioner reading these claims has no way to assess their validity or magnitude. If multi-token training provides a 5% relative accuracy improvement, that is a strong argument for adopting it in other document parsing models; if it provides 0.5%, it may not be worth the implementation complexity. If NoPE degrades accuracy by 3% but enables 2× longer sequences, the tradeoff needs to be quantified to inform design decisions. By presenting these as findings without evidence, the paper asks the reader to accept methodological claims on faith, which undermines trust in the reported results — if these claims are unsubstantiated, what other design choices described as "findings" might be similarly unsupported?

The absence of evidence also prevents the community from building on these techniques. If multi-token training genuinely improves single-token accuracy, this would be an impactful finding for the broader VLM training literature. But without quantifying the effect (at what values of m, on which metrics, under what training conditions), other researchers cannot replicate or extend the finding. The claim remains an anecdote rather than a contribution.

What evidence exists in the paper. None. The NoPE claim references external work (Kazemnejad et al., 2023; Zuo et al., 2025) for theoretical justification, but provides no internal validation. The multi-token claim provides no external citations for the accuracy improvement and no internal validation. Both claims appear in Section 2.1 with no corresponding experiment in Section 4. The paper's benchmarks (Tables 2–7) evaluate only the final model configuration, making it impossible to isolate the effect of either design choice.

Mitigation status. The paper does not acknowledge the absence of supporting evidence for these claims. This is a peer-review-level weakness — a technical report making empirical claims should either provide the supporting experiments or qualify the claims as design hypotheses rather than findings. A minimal mitigation would be to state something like "we hypothesize based on preliminary experiments that..." or "prior work suggests that..." rather than presenting these as experimentally validated results. The paper's current phrasing ("we find that") implies empirical validation that is not provided.

6.4 The TC Variant's Reading Order Improvement Is Unexplained and Confounded

The assumption or constraint. Nemotron-Parse-TC demonstrates substantially better reading order than the base model on OmniDocBench: 0.048 vs. 0.066 normalized edit distance (Table 4), a 27% relative improvement. The paper attributes this to the TC variant's different output convention — it "includes non-reading-order (floating) elements... within the natural ordering of the page" (Section 2.2.2), while the base model places floating elements at the end of the output. This attribution implies the improvement comes from the output format change, not from the token compression itself.

The consequence. The paper presents these as two changes in a single model variant: token compression (3201 → 833 vision tokens) AND a different output ordering convention (inline floating elements). Because both changes are applied simultaneously, the reader cannot determine which one causes the reading order improvement. Several competing explanations are possible:

1. Output convention effect: The inline ordering simply matches the evaluation metric's expectation better, and the base model would perform equally well if it used the same convention.
2. Compression effect: The coarser 833-token visual representation forces the model to learn more robust layout understanding that generalizes better, and the output convention change is incidental.
3. Training data effect: The TC variant may have been trained on additional data with inline ordering annotations that the base model did not receive, and the compression is incidental.
4. Interaction effect: The combination of compression and format change is necessary — neither alone would produce the improvement.

Without an ablation that isolates these factors (e.g., testing the base model with inline ordering, testing the TC variant with end-of-page ordering, testing an intermediate-resolution model), the paper's attribution of the improvement to the output convention is a hypothesis, not a demonstrated causal relationship.

What evidence exists in the paper. The only evidence is the OmniDocBench reading order scores in Table 4 (0.066 base, 0.048 TC) and the textual description in Section 2.2.2. The paper does not report whether the base model was capable of producing inline ordering (i.e., whether the output convention is a model capability limitation or an evaluation protocol choice). It does not report whether both variants were trained on identical data with different output conventions, or on different data. The GOT benchmark reading order scores (Table 3) show both variants achieving identical edit distance (0.014), which is inconsistent with the OmniDocBench pattern — if the TC variant has fundamentally better reading order, why does it not manifest on the GOT benchmark? This inconsistency further suggests that benchmark-specific factors (output format expectations, document types) may drive the difference rather than a genuine reading order capability improvement.

Mitigation status. The paper does not acknowledge this confound. The reading order improvement is presented as a feature of the TC variant without explaining its cause or demonstrating that the base model cannot achieve the same improvement through an output format change alone. For a practitioner deciding whether to deploy the base model or TC variant, the reading order difference is one of the most salient factors — the TC variant improves on the metric where the base model is weakest relative to pipeline systems — but the practitioner cannot determine whether the improvement comes from the model itself or from a simple post-processing change that could be applied to the base model's output. A clean experiment isolating the output convention from the compression would resolve this and substantially strengthen the paper's practical guidance.

6.5 No FLOPs-Normalized or Latency-Normalized Comparison Against Larger Models

The assumption or constraint. The paper positions Nemotron-Parse-1.1 as a "lightweight" model (885M parameters) that is "competitive" with much larger systems. The OmniDocBench table (Table 4) includes models ranging from compact (SmolDocling, DeepSeek-OCR-Tiny) to massive (Qwen2.5-VL-72B, InternVL3-78B at 72B–78B parameters). However, all comparisons are purely quality-based: normalized edit distance scores are reported without any accounting for the computational cost of achieving those scores. The only efficiency metric reported is throughput in tokens/second for Nemotron-Parse itself (Table 8), with no comparable numbers for competing models.

The consequence. The "lightweight" claim is meaningful only if the quality-per-FLOP or quality-per-second is competitive. A model with 885M parameters achieving 0.131 OmniDocBench overall error might be more efficient than a 72B-parameter model achieving 0.214 (Qwen2.5-VL-72B) — but it might be less efficient than DeepSeek-OCR-Gundam, which achieves 0.127 at 795 vision tokens with an unreported decoder size. Without FLOPs or latency normalization, the reader cannot determine whether Nemotron-Parse-1.1 is the most efficient model at its quality level, or whether a smaller model with slightly worse quality might offer better throughput-per-accuracy.

This matters acutely for the TC variant. The paper reports that TC provides "a 20% speed improvement" (3800 → 4500 tokens/second). But does this speedup make TC more efficient than DeepSeek-OCR-Gundam (795 tokens, 0.127 overall) or DeepSeek-OCR-Base (256 tokens, 0.137 overall)? These models use fewer vision tokens than TC's 833, potentially offering lower latency, but their decoder sizes are not reported in the paper. Without knowing the total inference FLOPs of competing models, the practitioner cannot place Nemotron-Parse-TC on the efficiency frontier.

What evidence exists in the paper. The "Tokens" column in Table 4 provides a partial efficiency metric — vision token count affects cross-attention FLOPs, which scale linearly with token count for a given decoder. By this metric, Nemotron-Parse-TC (833 tokens) is in the same range as DeepSeek-OCR-Gundam (795) and achieves comparable quality (0.129 vs. 0.127). However, vision token count ignores:

• Decoder parameter count: DeepSeek-OCR's decoder size is not reported in Table 4 (the paper cites Wei et al., 2025, but does not reproduce their decoder specifications). If DeepSeek-OCR-Gundam has a 1B-parameter decoder to Nemotron-Parse's 256M, the total FLOPs comparison shifts substantially.
• Output sequence length: A model that produces verbose output (e.g., including extra formatting tokens) incurs more decoder self-attention FLOPs than a compact-output model, even at the same vision token count. Output lengths are not reported.
• Hardware and batch size: Throughput depends on GPU architecture, batch size, and memory bandwidth. Table 8 reports throughput on "a single H100 GPU" for Nemotron-Parse only. No comparable single-GPU numbers are available for competing models.

Mitigation status. The paper does not attempt FLOPs-normalized or latency-normalized comparisons against competitors. The throughput numbers in Table 8 are useful for comparing the base and TC variants internally, but do not extend to cross-model efficiency comparisons. A FLOPs-normalized scatter plot (OmniDocBench error vs. estimated inference FLOPs) or a latency-quality tradeoff curve across models would substantially strengthen the "lightweight" and "competitive" claims. The paper's decision to report vision token counts alongside quality scores is a step toward efficiency comparison but is incomplete without decoder sizes and an acknowledgment of what is omitted.

7. Implications and Future Directions

How This Work Changes the Landscape

Nemotron-Parse-1.1 is not a paradigm shift — it does not introduce a fundamentally new architecture, training objective, or theoretical framework for document parsing. Rather, it is best understood as a demonstrative reframing of the document parsing problem around training data engineering and compute-efficiency characterization, with two specific contributions that shift how practitioners should think about building OCR systems.

The first shift is methodological and concerns how training data for document parsing should be generated. The NVpdftex pipeline (Section 3.1) demonstrates that intercepting the typesetting engine at compilation time — capturing bounding boxes, semantic classes, and reading order directly from LaTeX's internal layout representation — produces training data of substantially higher quality than the standard approach of rendering first and annotating after. This is not merely a "better data pipeline"; it is a reversal of the annotation paradigm. Before this work, the dominant approach (inherited from Nougat, Blecher et al., 2024) was to render the document, then apply a chain of lossy conversions (LaTeX → HTML → markdown) to recover structure from the rendered output. NVpdftex shows that you can avoid the recovery problem entirely by capturing structural information at the source — when the typesetting engine knows exactly where every character is, which semantic role every element plays, and what order the elements should be read in.

This matters because it changes the calculus for what data is worth generating. Prior to this work, a practitioner wanting high-quality document parsing data with bounding boxes, semantic classes, and reading order faced an expensive choice: pay for human annotation (like DocLayNet), or accept the quality loss from lossy automatic conversion pipelines. NVpdftex demonstrates a third path: for the substantial fraction of professional documents that originate in LaTeX (academic papers, technical reports, many preprints, some books), pixel-accurate annotations are essentially free — they can be extracted at compilation time with zero annotation error. The 8.3M pages produced by this pipeline (Table 1) represent a scale of high-quality annotation that would be economically infeasible through human labeling. The open-source release of the pipeline extends this capability to any organization with a LaTeX document corpus.

The second shift is empirical and concerns the relationship between visual token count and document parsing quality. Before this work, vision token counts in document parsing models ranged from 64 to 6,790 (Table 4) with no systematic characterization of the tradeoff. Practitioners chose token counts based on the vision encoder's default patch grid and whatever ad-hoc pooling felt reasonable — there was no evidence base for deciding whether 800 tokens or 3,200 tokens was appropriate for a given accuracy target. Nemotron-Parse-1.1 and its TC variant provide the most thorough single-paper characterization of this tradeoff to date: across four independent benchmarks (Tables 2, 3, 4, 6) and multiple metrics (F1, WER, edit distance, TEDS, reading order), reducing vision tokens from 3,201 to 833 produces negligible average quality loss (0.129 vs. 0.131 OmniDocBench overall error) while delivering an 18% throughput improvement (4,500 vs. 3,800 tokens/second, Table 8). The TC variant sometimes outperforms the base model on specific metrics (OmniDocBench reading order: 0.048 vs. 0.066; OmniDocBench table metrics: 84.73/91.44 TEDS/S-TEDS vs. 82.68/89.06).

This result changes the default assumption for practitioners. The natural prior — that higher visual resolution always improves document understanding — turns out to be wrong for this task at this model scale. The finding that 833 tokens are sufficient for competitive document parsing, and that the 3,201-token base model may actually overfit to training-data-specific output conventions (the end-of-page floating element ordering that the TC variant corrects), suggests that practitioners should default to aggressive vision token compression and only increase resolution if specific benchmarks show quality degradation. This is an actionable design rule that did not exist before this paper.

The paper also contributes a diagnostic finding about reading order that has implications beyond Nemotron-Parse itself. The TC variant's improved reading order (0.048 vs. 0.066 on OmniDocBench) — attributed to inline placement of floating elements rather than end-of-page placement — reveals that the output convention (how the model orders extracted elements) can matter as much for benchmark performance as the model's underlying extraction quality. This is a reminder that document parsing benchmarks measure conformance to specific output formats, not just content accuracy, and that model comparisons can be sensitive to format-matching choices that are orthogonal to capability. For benchmark designers, this suggests that reading order evaluation should be robust to different but equally valid ordering conventions; for model developers, it suggests that output format should be treated as a first-class design choice, not an afterthought.

The paper does not resolve the ongoing tension between pipeline and end-to-end approaches. The OmniDocBench results (Table 4) show pipeline systems (Dolphin: 0.356, Marker: 0.296, Mathpix: 0.191) and end-to-end models (Nemotron-Parse-TC: 0.129, DeepSeek-OCR-Gundam: 0.127) both occupying the top of the leaderboard, with no clear winner. Nemotron-Parse-1.1 narrows the gap from the end-to-end side — its 0.131 overall score is substantially better than earlier end-to-end models like Nougat (0.452) and SmolDocling (0.493) — but it does not demonstrate that end-to-end models can match the best pipeline systems on all sub-tasks simultaneously. The paper's contribution is to show that a single lightweight end-to-end model can be competitive across all sub-tasks, which is a practical advance even if it is not a theoretical resolution of the pipeline-vs-end-to-end debate.

What research directions become more attractive as a result of this work:

• Data generation through typesetting engine interception (not just LaTeX — the principle extends to any engine that performs layout: HTML/CSS renderers for web pages, InDesign for magazine layouts, Word for office documents). The NVpdftex approach shows that capturing structural information at layout time is superior to recovering it post-rendering; extending this to non-LaTeX typesetting engines could dramatically expand the range of documents for which high-quality training data can be automatically generated.
• Token compression as a deliberate design axis, not an afterthought. The TC variant's near-lossless compression at 4× reduction suggests that even more aggressive compression (e.g., 400 tokens, 200 tokens) should be systematically explored to find the true floor of required visual resolution for document parsing. The TC variant at 833 tokens achieving 0.129 overall error while DeepSeek-OCR-Gundam at 795 tokens achieves 0.127 — and DeepSeek-OCR-Base at only 256 tokens achieves 0.137 — suggests the knee of the curve may be well below 800 tokens.
• Output format conventions as an evaluation design problem. The reading order metric sensitivity to floating element placement reveals that document parsing benchmarks need to account for multiple valid output formats. A benchmark that penalizes a model for placing tables at the end of the page (a perfectly reasonable convention for some downstream tasks) is measuring format conformance, not extraction quality. This suggests a research direction in format-robust evaluation metrics that can assess content and structure independently of output convention.

What research directions become less attractive as a result of this work:

• Scaling vision encoder size for document parsing. The OmniDocBench results show that models with 72B–78B parameters and 3,949–6,790 vision tokens (Qwen2.5-VL-72B: 0.214 overall; InternVL3-78B: 0.218) are substantially worse than Nemotron-Parse-1.1 at 885M parameters and 3,201 tokens (0.131). Raw scale does not compensate for training data quality and task specialization. This suggests that research effort is better spent on data engineering and task-specific architecture choices than on scaling generalist VLMs for document parsing.
• Complex multi-stage pipeline architectures for general document parsing, at least for throughput-sensitive applications. The paper demonstrates that a single end-to-end model can be competitive across text, formulas, tables, and reading order simultaneously — the traditional argument for pipelines (each stage optimized for its subtask) weakens when a unified model achieves balanced performance across all subtasks. Pipeline research remains valuable for settings where peak sub-task performance matters more than throughput or deployment simplicity, but the cost-benefit calculus has shifted.

Follow-Up Research This Work Enables

Document-type-stratified evaluation to characterize the NVpdftex domain gap. The paper's training data is dominated by LaTeX-source academic documents (8.3M NVpdftex samples, plus 9.5M Wikipedia documents also rendered via LaTeX, totaling ~78% of the training blend). The OmniDocBench and GOT benchmarks include diverse document types, but the paper does not report per-document-type breakdowns. A follow-up study could take the publicly released Nemotron-Parse-1.1 weights, classify a diverse test set (OmniDocBench or a custom collection) by document type (academic paper, financial report, legal contract, magazine layout, scanned form, photograph of a receipt, web page screenshot), and measure performance per category. The hypothesis to test: Nemotron-Parse-1.1's quality degrades substantially on document types with no LaTeX-source representation in training (scanned forms, photographs, web pages) relative to LaTeX-adjacent types (academic papers, technical reports). If the degradation is large (e.g., >2× WER on photographs vs. academic papers), this would quantify the domain gap and motivate targeted data augmentation for those types. If the degradation is small, it would demonstrate that the supplementary data sources (DocLayNet, Common Crawl) effectively transfer NVpdftex-learned capabilities to non-LaTeX documents. This experiment requires only the released model weights and a document-type-labeled test set.

Systematic vision token compression sweep to map the quality-efficiency Pareto frontier. The paper evaluates exactly two compression ratios: 1× (base, 3,201 tokens) and 3.8× (TC, 833 tokens). The TC variant's near-lossless compression raises the question: how low can we go before quality degrades materially? A follow-up could train Nemotron-Parse variants at intermediate compression ratios (e.g., 2,400, 1,600, 1,200, 600, 400, 200 vision tokens) by adjusting the pixel-shuffle downsampling factor, keeping all other aspects of training identical. Evaluating this sweep on OmniDocBench would produce a quality-vs-token-count curve that identifies the knee — the point at which further compression causes disproportionate quality loss. The paper's existing results hint that the knee may be below 833 tokens (since DeepSeek-OCR-Base at 256 tokens achieves 0.137 overall, only 0.008 worse than TC at 0.129), but a controlled within-model-family sweep is needed to isolate the token count effect from architectural and training differences. The practical payoff is a concrete recommendation: "for document parsing with 885M-parameter encoder-decoder models, use X vision tokens — below this, quality degrades rapidly; above this, tokens are wasted."

Ablation of multi-token training on single-token inference accuracy with varied m. The paper claims (Section 2.1.2) that multi-token training improves single-token inference accuracy but provides no evidence. A targeted follow-up would train identical Nemotron-Parse architectures with m = 1, 2, 3, 4, and 6 simultaneous token predictions (controlling for total training compute by adjusting step count), then evaluate all models in single-token inference mode on OmniDocBench and the internal 789-page benchmark. The key measurement: the difference in single-token accuracy between m=1 (standard training) and m>1 (multi-token auxiliary objective). Additional measurements: (a) multi-token inference quality at each m (to characterize the speed-quality tradeoff when actually using the multi-token heads), (b) per-step accuracy of the k-th predicted token for k=2,...,m (to characterize error accumulation), (c) whether the benefit saturates or peaks at some m. If the accuracy benefit is real and non-trivial (>2% relative on OmniDocBench), this would establish multi-token training as a recommended practice for document parsing model training, not just a throughput optimization. If the benefit is negligible or negative, the paper's claim should be retracted and the community can stop investing effort in multi-token auxiliary objectives for this task.

NoPE vs. learned positional embeddings head-to-head comparison with length generalization stress test. The paper claims that omitting positional embeddings (NoPE) "achieves comparable accuracy" while enabling "inference with significantly longer context lengths" (Section 2.1.1), but provides no comparison. A clean experiment would train two Nemotron-Parse variants — one with NoPE and one with standard learned absolute positional embeddings (or rotary position embeddings) — on identical data with a fixed training sequence length (e.g., 4,096 tokens), then evaluate both on: (a) in-distribution-length documents (to test the "comparable accuracy" claim), and (b) out-of-distribution-length documents (6,000, 8,000, 12,000 tokens — to test the length generalization claim). The key measurement for (b) is the slope of quality degradation as sequence length exceeds the training maximum. For the positional embedding variant, degradation is expected due to position interpolation or out-of-distribution position indices; for NoPE, the paper predicts minimal degradation. If the NoPE variant indeed maintains quality on long sequences while the positional embedding variant degrades, this would provide strong evidence for NoPE in document parsing and, more broadly, in any vision-to-sequence task where output length varies substantially. If both variants degrade similarly, or if NoPE underperforms even at in-distribution lengths, the paper's design choice loses its justification.

Prompt-conditioning ablation: single-format vs. multi-format training with capacity control. The paper's training methodology — conditioning the model on prompt tokens to handle heterogeneous annotation formats — is argued to enable knowledge transfer across formats, but this is untested. A controlled experiment would compare three training setups with equal total model capacity: (a) a single multi-format model trained with prompt conditioning on all data sources (the Nemotron-Parse approach), (b) three separate single-format models (one for plain text, one for markdown, one for markdown+boxes+classes), each trained only on data with its target format, with decoder capacity scaled so that total parameters equal the multi-format model, and (c) a multi-task model with separate output heads for each format rather than prompt conditioning. Evaluating all setups on each output format separately would measure: (1) whether the multi-format model matches or exceeds single-format specialists on each format (testing for positive transfer), (2) whether prompt conditioning outperforms separate output heads (testing the conditioning mechanism vs. architectural alternatives), and (3) whether the multi-format model shows benefits on sparsely annotated formats (e.g., does training with NVpdftex's rich annotations improve plain-text extraction on Common Crawl data?). If positive transfer is demonstrated, this validates the paper's core training methodology and provides a template for other grounded generation tasks with heterogeneous supervision. If negative transfer is found (the multi-format model underperforms specialists), it would suggest that the paper's strong results come from data scale and quality, not from the prompt-conditioning design, and practitioners should train separate models for each output format.

Practical Applications and Downstream Use Cases

Batch preprocessing for retrieval-augmented generation (RAG) over large document corpora. Organizations with millions of PDFs — legal firms with case archives, pharmaceutical companies with research paper collections, financial institutions with regulatory filings — need to convert image-based documents into structured text for indexing and retrieval. Nemotron-Parse-1.1-TC's throughput of 4,500 tokens/second on a single H100 (Table 8), translating to approximately 5 pages/second, means a single GPU can process roughly 18,000 pages per hour or 430,000 pages per day. For a corpus of 10 million pages, a modest 24-GPU cluster could complete preprocessing in under 24 hours. The specific benefit over alternatives is the combination of (a) structured output with bounding boxes and semantic classes — enabling layout-aware chunking that preserves section structure and table context, which plain-text OCR cannot provide — and (b) competitive accuracy (0.129 OmniDocBench overall error, Table 4) without requiring a multi-stage pipeline. For a RAG system answering "what were the Q3 revenues in the 2023 annual report?", the parser's ability to extract tables as structured LaTeX (TEDS 84.73 on OmniDocBench, Table 5) and preserve their position in reading order (0.048 edit distance, Table 4) means the retrieved chunk will contain a machine-readable table rather than garbled cell text, directly improving downstream LLM answer quality.

On-device document scanning with structured output for mobile applications. Mobile scanning apps (e.g., scanning a receipt, business card, or whiteboard photo) need OCR that runs within tight latency and memory budgets on phone-grade hardware. Nemotron-Parse-1.1's 885M parameters and 256M-parameter decoder are small enough to quantize (the paper releases fp32/bf16 weights, and INT8/INT4 quantization is a standard downstream step) for mobile deployment. The TC variant's 833 vision tokens (vs. 3,201) reduces both memory footprint (smaller cross-attention key-value cache) and compute (3.8× fewer cross-attention operations per generated token). For a receipt scanning application, the parser's semantic class output — distinguishing line items (Text) from totals (potentially classed as List-Item or a dedicated semantic class) from merchant headers (Page-Header) — enables automatic extraction of structured data (merchant name, date, line items, total) without post-hoc regex or heuristics. The specific numbers: at 5 pages/second on an H100 (Table 8), even with a 10× slowdown from mobile quantization and hardware constraints (a conservative estimate for phone-grade GPUs/NPUs), a single receipt image would process in ~2 seconds — acceptable for an interactive camera-scanning UX.

Training data generation for downstream document understanding models. A significant bottleneck in training document layout analysis models, table extraction models, or reading-order models is the cost of human annotation for bounding boxes, semantic classes, and reading order. An organization training a custom layout model for a specific document type (e.g., insurance claim forms) can use Nemotron-Parse-1.1 as a pseudo-labeling engine: run the model on unlabeled documents in MIP configuration (<output_markdown><predict_bbox><predict_classes>), obtain bounding boxes with semantic classes and reading order, filter low-confidence predictions (the paper's Common Crawl data augmentation uses "edit distance of the formatting-stripped output to the plaintext labels" as a quality filter — Section 3.1), and use the filtered outputs as training data for a specialized model. The specific advantage over using a generic detector (like a YOLO-based layout model) is that Nemotron-Parse provides text content, bounding boxes, semantic classes, and reading order simultaneously — four annotation types that would otherwise require four separate models or expensive multi-annotator human labeling. The paper's own bootstrapping approach (using "stage-1 trained Nemotron-Parse" to autolabel Common Crawl formatting, Section 3.1) validates this workflow. The quality of pseudo-labels can be estimated from the model's benchmark performance: F1 of 0.958 on the internal test set (Table 2) and OmniDocBench text error of 0.052 (Table 4) suggest that the majority of pseudo-labeled bounding boxes and text will be correct, with most errors concentrated on challenging cases (dense tables, unusual fonts, heavily stylized documents) that can be targeted for human correction.

When to Prefer This Method

The paper does not articulate an explicit decision rule or tradeoff matrix positioning Nemotron-Parse-1.1 against specific named alternatives for specific use cases. The benchmark comparisons (Tables 2–7) present quality metrics without corresponding latency, cost, or deployment-complexity comparisons that would enable a practitioner to choose between Nemotron-Parse-1.1 and, say, DeepSeek-OCR-Gundam or Gemini Flash 2.0. The closest the paper comes to a preference articulation is in the TC variant's positioning: it is designed "for applications that prioritize speed without majorly sacrificing output quality," "large-scale batch processing, edge deployments, or interactive systems where rapid response times are critical" (Section 1). But this is a preference for TC over the base model, not a preference for Nemotron-Parse over competing systems. The paper does not provide the data (FLOPs-normalized comparisons, latency measurements for competing models, cost-per-page estimates) that would be needed to construct a well-grounded decision rule. A forced "Prefer Nemotron-Parse when X, prefer DeepSeek-OCR when Y" matrix would be speculation beyond what the paper's experiments support.