DeepSeek-OCR 2: Visual Causal Flow

ArXiv: 2601.20552

🎯 Pitch

Instead of the usual rigid raster-scan order, this vision encoder learns to reorder image patches into a semantically meaningful sequence before the language model sees them, mimicking how human eyes follow causal visual logic. By replacing CLIP with a regular language model that peers at the whole image and then decides what to attend to next, the architecture achieves state-of-the-art OCR using drastically fewer visual tokens. This design reframes 2D image understanding as a clean two-stage 1D causal reasoning problem.

---

1. Executive Summary

This paper introduces DeepEncoder V2, a novel vision encoder that replaces the conventional CLIP component with a compact LLM-style architecture to enable visual causal flow—the ability to semantically reorder visual tokens before passing them to a language decoder. Evaluated on document OCR using OmniDocBench v1.5, DeepSeek-OCR 2 (built with DeepEncoder V2 and a DeepSeek-MoE decoder) achieves 91.09% overall accuracy, a 3.73% absolute improvement over its predecessor, while using a maximum of only 1,120 visual tokens (matching Gemini-3 Pro's budget). The architecture decomposes 2D image understanding into two cascaded 1D causal reasoning stages—the encoder causally reorders visual tokens via learnable queries (analogous to how human vision sequentially fixates along semantic rather than raster-scan paths), and the LLM decoder performs autoregressive reasoning over the reordered sequence—establishing that LLM-style architectures can serve as effective vision encoders, though current gains are most pronounced on structured visual reading logic tasks where spatial-semantic mismatch between raster ordering and natural reading order is most acute.

2. Context and Motivation

The Core Problem: Raster-Scan Ordering Contradicts Visual Semantics

The fundamental problem this paper tackles is architectural: conventional vision-language models (VLMs) process image patches in a fixed, spatially determined order, but images possess semantic structure that does not align with spatial coordinates. When a VLM like DeepSeek-OCR, LLaVA, or Qwen-VL encodes an image, it invariably follows the same pipeline: divide the image into patches, flatten them in raster-scan order (top-left to bottom-right), and feed the resulting 1D sequence into a transformer with positional encodings. This ordering is rigid, predetermined, and—crucially—imposed without any regard for the semantic content of the image.

The paper illustrates this mismatch with a deceptively simple example: tracing a spiral. When the human eye follows a spiral pattern, each fixation causally depends on the previous one—the eyes move along the spiral's curve, following its inherent semantic logic. A raster-scan encoding of the same spiral image would slice through the curve arbitrarily, placing spatially adjacent patches next to each other in the sequence even when they belong to entirely different parts of the spiral and carry no meaningful semantic relationship. The spatial proximity that raster ordering exploits is an artifact of the 2D grid, not a reflection of visual content logic.

This mismatch is particularly acute in document images, which the paper adopts as its primary experimental testbed. Documents contain structured elements—multi-column layouts, tables, formulas, figures with captions, footnotes—whose natural reading order bears little resemblance to a simple left-to-right, top-to-bottom scan. Consider a two-column academic paper layout: raster-scan processing would interleave tokens from the left and right columns at each row line, producing a sequence that jumps back and forth between semantically unrelated content. Humans, by contrast, follow a causally-informed scanning pattern: read the abstract, then introduction, then figure captions, then the main text column-by-column, dynamically re-fixating based on understood structure. This gap between the rigid spatial ordering of vision encoders and the flexible semantic ordering of human reading is what the paper identifies as "unwarranted inductive bias" (Section 1).

Why This Problem Matters

The significance of this problem extends across multiple levels—from the practical deployment of OCR systems to deep theoretical questions about how visual understanding should be modeled architecturally.

Practical impact on document understanding. Optical Character Recognition in the LLM era has evolved far beyond simple text extraction. Modern OCR systems like DeepSeek-OCR serve as critical infrastructure in two production pipelines: (1) online OCR services that feed document images into LLMs for question-answering, summarization, and reasoning; and (2) pretraining data pipelines that process millions of PDF documents into training data for next-generation language models. In both settings, a model that misorders visual information produces cascading failures downstream. If the encoder feeds the LLM a jumbled sequence where a table caption precedes its content, a sidebar interrupts the main text flow, or footnote references appear disconnected from their contexts, the LLM cannot recover—it processes the sequence linearly and trusts the ordering it receives. The paper's practical readiness evaluation (Section 5.3) makes this explicit: a key production metric is repetition rate, where garbled reading order causes the LLM to generate repetitive, semantically incoherent text. DeepSeek-OCR 2's reduction of repetition rate from 6.25% to 4.17% on online user images (Table 4) directly reflects the real-world impact of better visual ordering.

Theoretical significance for VLM architecture design. At a deeper level, the raster-order problem raises a fundamental question about the VLM architecture paradigm: can genuine 2D visual understanding emerge from a model that inherently processes information in 1D causal chains? This is the architectural puzzle the paper confronts. LLMs are trained on 1D sequential data with causal (unidirectional) attention—they are fundamentally autoregressive engines that predict the next token based on prior tokens. Images are 2D structures where concepts relate non-sequentially (a figure relates to its caption, a table cell relates to its row and column headers, a diagram relates to surrounding explanatory text). The predominant solution in the field—flatten 2D into 1D with spatial positional encodings and apply bidirectional attention in the encoder, then pass to the 1D causal decoder—has proven remarkably effective but may be fundamentally limited. Bidirectional attention in the encoder allows every patch to attend to every other patch, which captures spatial relationships but does not impose any semantic ordering. The resulting sequence that reaches the decoder is still ordered by spatial coordinates, not by visual semantics.

The paper's framing of this as a question about "whether 2D image understanding can be effectively achieved through two-cascaded 1D causal reasoning structures" (Section 1) is an architectural hypothesis of genuine depth. If successful, it would suggest that 2D reasoning can be decomposed into: (1) a semantic reordering stage that transforms spatial structure into causal-logical structure, and (2) a sequential reasoning stage that processes the causally-ordered representation. This decomposition, if generalizable, would provide a blueprint for unifying visual and linguistic processing under a single causal reasoning paradigm—a step toward the "native multimodality" the paper envisions in Section 6.2.

Computational efficiency at scale. The visual token budget matters enormously for practical deployment. Each visual token fed to the LLM decoder incurs quadratic attention cost. DeepEncoder V2 constrains its output to 256–1,120 visual tokens (Section 3.2.3), with the upper bound matching Gemini-3 Pro's maximum visual token budget. This is achieved through a 16× compression ratio in the vision tokenizer (SAM-base + convolutional layers) and the multi-crop strategy. Maintaining high compression while improving semantic ordering is non-trivial—more compression typically means lost information, but better ordering could mean more efficient use of the remaining tokens. The paper's simultaneous improvement in performance (91.09% vs. 87.36%) while reducing the maximum token budget (1,120 vs. 1,156) demonstrates that semantic reordering can compensate for, and even overcome, information loss from token compression, since better-ordered tokens allow the LLM to extract meaning more efficiently from the same number of tokens.

Where Prior Approaches Fall Short

The paper identifies a landscape of prior work that addresses visual processing with three broad approaches, each with specific limitations that motivate DeepEncoder V2.

Standard encoder-projector-LLM architectures with spatial ordering. The dominant VLM paradigm—exemplified by LLaVA, Qwen-VL, InternVL, and DeepSeek-OCR itself—uses a vision encoder (typically CLIP ViT) to produce visual tokens, a projector to align them with the LLM's embedding space, and an LLM decoder to generate text conditioned on the visual prefix. All of these architectures, without exception, maintain the raster-scan ordering of visual tokens established by the vision encoder's patch embedding. The encoder applies bidirectional attention, which allows each token to attend to every other token regardless of position—this captures spatial relationships effectively. But the sequence representation that emerges—the specific order in which tokens are arranged when passed to the decoder—remains determined by the initial spatial coordinates, not by the semantic content the encoder has processed.

This is a subtle but crucial point: bidirectional attention captures relationships between tokens, but does not reorder them. The decoder receives a sequence where token 0 is always the top-left patch, token 1 is the patch to its right, and so on. Any semantic structure—a table, a multi-column layout, a reading order—must be inferred by the decoder from these spatially-ordered tokens, using the content and positional encodings to reconstruct the underlying visual logic. This places the entire burden of 2D-to-1D conversion on the decoder, which must simultaneously understand visual content and figure out its semantic ordering from a spatially-scrambled sequence.

Parallelized query approaches that compress but don't reorder. The paper's Related Work section (Sections 2.1–2.2) explicitly situates its contribution against two established uses of learnable queries in computer vision: DETR (Carion et al., 2020) and BLIP-2's Q-former (Li et al., 2023). Both use parallelized queries—fixed sets of learned embeddings that interact with visual features via cross-attention—but for fundamentally different purposes and with fundamentally different ordering properties.

DETR introduced object queries for detection: 100 learned query vectors that each decode a single object's bounding box and class. These queries interact through bidirectional self-attention (all queries attend to all other queries simultaneously), meaning there is no notion of order among them. Each query independently extracts its object's features from the image feature map. The queries are parallelized for efficiency (replacing the serial decoding of earlier transformer detection methods) but are not sequenced—there is no causal dependency, no ordering, and no semantic flow from one query to the next.

BLIP-2's Q-former extended this concept to vision-language alignment, using 32 learned queries that compress hundreds of CLIP visual tokens into a compact representation for LLM consumption. The Q-former also uses bidirectional self-attention among queries. Like DETR, the queries operate in parallel without causal dependencies. The compression is effective—32 tokens can represent the visual content of the image—but the compression is order-agnostic. The sequence of compressed tokens fed to the LLM bears no particular semantic relationship to reading order; it is simply a fixed ordering of the learned query embeddings.

DeepEncoder V2 departs from both of these prior approaches by introducing causal attention among queries. This is the critical architectural distinction: queries no longer attend to all other queries simultaneously (bidirectional) but instead attend only to preceding queries (causal triangular mask). This imposes sequential dependency—query i can "see" queries 0 through i-1 but not queries i+1 through n-1. This causal structure is what enables semantic reordering: each query's representation is causally conditioned on the previous queries' representations, creating a sequential flow where later queries can build on the visual information distilled by earlier queries. The paper argues this mirrors the causal flow of human visual fixations, where each subsequent fixation is informed by the cumulative understanding built from prior fixations.

LLM-based multimodal initialization without visual reasoning. Section 2.3 discusses approaches that initialize visual models from LLM pretrained weights—Fuyu, Chameleon, VALL-E. These works demonstrate that LLM pretrained weights contain useful representations for non-text modalities, but they do not address the ordering problem. Fuyu, for instance, feeds raw image patches directly into an LLM without any dedicated vision encoder, but the patches are still processed in raster-scan order. The initialization strategy is valuable (LLM weights provide better starting points for visual tasks than random initialization) but the fundamental limitation—spatial ordering imposed on semantic content—remains.

What makes the gap acute. There is a genuine tension in the VLM architecture that prior work has not directly addressed: the vision encoder uses bidirectional attention (which is order-agnostic—attention weights depend on key-query similarity, not position), yet the LLM decoder expects tokens arranged in a causally meaningful order. This creates a representational mismatch. The encoder produces tokens where "nearby" in the sequence means "spatially adjacent in the image," but the decoder interprets "nearby" as "semantically related and causally sequential." The paper's insight is that this mismatch can be resolved by introducing an intermediate stage that performs causal reordering of visual information before it reaches the decoder—a stage that converts spatial adjacency into semantic sequentiality.

How This Paper Positions Itself

Building directly on DeepSeek-OCR. The paper positions DeepSeek-OCR 2 as an evolutionary step from its predecessor. It inherits the overall architecture (encoder-decoder with MoE decoder), the vision tokenizer (SAM-base + convolutional layers achieving 16× compression), the training data engine (OCR 1.0, OCR 2.0, general vision data with 80% OCR mixture), and the multi-stage training pipeline. The key difference is singular and targeted: replace the CLIP component in the encoder with an LLM-style architecture equipped with dual-stream attention (bidirectional for visual tokens, causal for query tokens). This is not a complete redesign but a focused intervention on the specific component responsible for ordering—the part of the encoder that transforms compressed visual features into tokens ready for LLM consumption.

This positioning allows the paper to attribute performance improvements cleanly to the architectural change. The training data sources are "minimal differences" (Section 4.1) from DeepSeek-OCR, making comparison valid. Experiments consistently compare against DeepSeek-OCR (Tables 1, 2, 3, 4), isolating the effect of DeepEncoder V2.

Bridging computer vision paradigms to language model paradigms. The paper explicitly draws connections between its approach and established computer vision techniques (DETR queries, Q-former compression) while simultaneously embedding the architecture within language model conventions (decoder-only transformer, causal attention, prefix-based visual injection). The dual-stream attention mask (Figure 5, Equation 1) is the clearest expression of this bridging: the left block is a ViT-style bidirectional mask ("CLIP's global modeling capability"), and the right block is a decoder-LLM-style causal triangular mask. By concatenating these, the architecture grafts causal reasoning capabilities onto an otherwise standard visual encoding backbone.

Situating relative to prior query-based methods. The paper distinguishes its queries from DETR's and Q-former's along two axes:

• Architectural role: DETR queries decode object instances; Q-former queries compress visual tokens; DeepEncoder V2 queries reorder visual information causally. The function is fundamentally different from both predecessors—not detection, not compression, but semantic sequencing.

• Attention pattern: Both DETR and Q-former use bidirectional self-attention among queries (all-to-all). DeepEncoder V2 uses causal self-attention (each query to preceding queries only). This apparently small change in the attention mask has profound implications for the model's behavior: it forces the model to learn an ordering over visual information by making each query's representation dependent on the sequence of preceding queries.

• Query cardinality: Both DETR (100 queries) and Q-former (32 queries) use far fewer queries than there are visual tokens—they compress. DeepEncoder V2 uses equal cardinality ($n = m$ in Equation 1, Section 3.2.2) between causal flow queries and visual tokens. This is explicitly justified as providing "sufficient capacity for re-fixation"—each visual token gets a corresponding causal query that can attend to and potentially re-express its information in a causally-ordered sequence. The cardinality is not about compression (which is handled by the earlier vision tokenizer) but about ensuring no information is lost during the reordering transformation.

Defining a new research direction: cascaded 1D causal reasoning for 2D understanding. The most ambitious positioning in the paper is the framing in Section 6.1: the encoder-decoder cascade represents two stages of 1D causal reasoning that together approximate 2D understanding. The encoder performs "reading logic reasoning"—causally reordering visual tokens based on semantic content—and the decoder performs "visual task reasoning" over the reordered sequence. This decomposition is positioned as a potential "breakthrough toward genuine 2D reasoning" and as a novel architectural paradigm, not just an incremental OCR improvement.

The paper is cautious in this claim—it explicitly notes "achieving this goal remains a long journey" and identifies a concrete limitation: "to enable multiple re-examinations and multi-hop reordering of visual content, we may need substantially longer causal flow tokens than the original visual token sequence." This suggests the current implementation (equal cardinality, single pass) is a first step, not a complete solution.

Practical positioning on visual token budget. An important strategic choice in the paper's positioning is the constraint on visual tokens (256–1,120). The lower bound matches DeepSeek-OCR's tokenization of 1024×1024 images; the upper bound matches Gemini-3 Pro's maximum visual token budget. This is not an arbitrary constraint—it positions DeepSeek-OCR 2 as competing in the "high-compression, high-efficiency" regime that matters for production deployment. Models like InternVL3 and GPT-4o use over 6,000–7,000 visual tokens (Table 1), which is computationally expensive at scale. DeepSeek-OCR 2 deliberately operates at the low end of the visual token budget spectrum, aiming to demonstrate that semantic reordering can compensate for compression—intelligence in the ordering reduces the need for brute-force token quantity.

Groundwork for native multimodality. Section 6.2 positions the LLM-style encoder as a step toward unified omni-modal encoding. The vision is a single encoder with shared parameters (projection matrices, attention mechanisms, FFNs) that processes multiple modalities through modality-specific learnable query embeddings. Text, audio, and images would all flow through the same encoder architecture, differing only in their query parameters. This positions DeepEncoder V2 not just as an OCR improvement but as an architectural validation that LLM-style encoders work for non-text modalities—a necessary prerequisite for building truly unified multi-modal models. The paper explicitly connects this to the LLM community's infrastructure optimizations (MoE architectures, efficient attention mechanisms like DeepSeek-V3's), suggesting that adopting LLM-style encoders for vision would allow visual models to inherit the massive engineering investment in efficient LLM training and inference.

3. Technical Approach

This is primarily a systems-design paper whose core idea is that an LLM-style decoder architecture—when equipped with a dual-stream attention mask that applies bidirectional attention to visual tokens and causal attention to a parallel stream of learnable queries—can acquire the ability to semantically reorder visual information before it reaches the language decoder, replacing the rigid raster-scan ordering that all conventional vision-language models inherit from patch embedding.

3.1 Reader Orientation

The system is a vision-language model specialized for document reading that replaces the traditional CLIP vision encoder with a compact language model (Qwen2-0.5B) repurposed to function as a visual causal reorderer. Instead of processing image patches in a fixed spatial order and passing them directly to the LLM decoder, it first runs them through an encoder stage where a set of learnable "causal flow queries"—equal in number to the visual tokens—progressively attend to all visual tokens and to preceding queries through a customized attention mask, producing a semantically reordered sequence that guides the LLM's subsequent autoregressive reasoning.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components chained together in a feedforward pipeline, plus a multi-crop pre-processing strategy:

1. Vision Tokenizer (SAM-base + convolutional layers, ~80M parameters): Takes a raw image (variable resolution) and produces compressed visual tokens at 16× compression ratio through window attention. This is inherited unchanged from DeepSeek-OCR.

2. DeepEncoder V2 (LLM-style encoder, Qwen2-0.5B ~500M parameters): The novel component. Takes compressed visual tokens and a matching set of learned causal flow queries as input. Processes them jointly through transformer layers with a dual-stream attention mask: visual tokens use bidirectional (ViT-style) attention, causal flow queries use causal (decoder-only LLM-style) attention, and queries can attend to all visual tokens plus preceding queries. Outputs only the causal flow queries (the reordered representation) while discarding the visual tokens.

3. Projection: Extracts the last $n$ tokens from the encoder output (the causal flow queries-only, where $n$ equals the number of visual tokens) and feeds them to the LLM decoder. This is a simple slicing operation, not a learned projection layer.

4. DeepSeek-MoE Decoder (3B parameters, ~500M active): Standard MoE language model that performs autoregressive text generation conditioned on the reordered visual tokens and a text prompt. Inherited unchanged from DeepSeek-OCR.

Multi-crop strategy (pre-processing): Before encoding, images are processed at multiple resolutions. A global view at 1024×1024 uses 256 causal flow queries. Local crops at 768×768 use 144 causal flow queries each, with 0–6 crops depending on image size. Total output tokens range from 256 (global only) to 1,120 (global + 6 local crops), matching Gemini-3 Pro's maximum visual token budget.

3.3 Roadmap for the Deep Dive

• First, the motivation for the architectural intervention—why the standard encoder's bidirectional attention is insufficient and what property causal attention adds—to establish the design goals before examining the mechanism.

• Second, the vision tokenizer (SAM-base + convolutional compression), since it determines the input resolution, compression ratio, and computational budget for everything downstream, and its 16× compression makes the subsequent global attention module feasible.

• Third, the LLM-style encoder core (Qwen2-0.5B instantiation), focusing on the dual-stream attention mask (the paper's central technical innovation), the causal flow query mechanism, and why equal query-to-visual-token cardinality matters.

• Fourth, the multi-crop resolution strategy, which is the practical bridge between the uniform query-cardinality design and the reality of variable-resolution input images.

• Fifth, the formal mathematical formulation (Equation 2 and the attention mask in Equation 1), which codifies the entire encoder-decoder pipeline as a single composable function.

• Sixth, the three-stage training pipeline and key hyperparameter configurations, including the rationale for freezing different components at different stages.

• Seventh, the design choices and alternatives considered—why cross-attention in an encoder-decoder structure failed, why equal cardinality matters, and why the prefix concatenation design works where alternatives didn't.

3.4 Detailed, Sentence-Based Technical Breakdown

The Fundamental Architectural Problem: Bidirectional Attention Doesn't Reorder

To understand what DeepEncoder V2 does, we need to first understand what the standard CLIP encoder does not do. In a conventional VLM like DeepSeek-OCR, the vision encoder (a CLIP ViT) applies bidirectional self-attention over all visual tokens simultaneously. Each token computes its output representation as a weighted sum of all other tokens' value vectors, where weights are determined by query-key similarity:

$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$

where $Q, K, V \in \mathbb{R}^{m \times d_k}$ are the query, key, and value matrices computed from the $m$ visual token embeddings, $d_k$ is the per-head dimension, and the softmax is applied row-wise over all $m$ tokens.

What it computes: for each of the $m$ visual tokens, a weighted combination of all $m$ token value vectors, where weights are determined by pairwise content similarity (not spatial position). The output is a permutation-equivariant transformation—if you shuffled the input tokens and applied the same attention, the output tokens would be shuffled in exactly the same way.

Why this is the problem: permutation equivariance means the model's representation of each token is invariant to the ordering of the other tokens in the sequence. But the LLM decoder is not permutation-invariant—it is an autoregressive model where token $i$'s representation can only depend on tokens $0$ through $i-1$. The ordering of visual tokens therefore matters enormously for what the decoder can extract. In a standard VLM, the ordering is fixed by raster scan (spatial coordinates), which means:

1. The encoder's rich global reasoning (achieved through bidirectional attention) produces token-level representations that capture cross-token relationships, but
2. These representations remain arranged in a spatial order that may bear no relationship to the semantic structure the encoder has discovered, and
3. The decoder must simultaneously infer both what the content is and how it should be ordered semantically—a dual burden that wastes decoder capacity.

DeepEncoder V2 intervenes at step 2: it introduces an intermediate processing stage between "understanding content through global attention" and "feeding tokens to the causal decoder," and this stage's explicit purpose is to convert spatial ordering into causal-semantic ordering.

The Vision Tokenizer: SAM-Base Compression for Computational Feasibility

Before visual tokens enter the LLM-style encoder, they must first be extracted from raw pixels and compressed to a tractable count. DeepEncoder V2 inherits this component from DeepSeek-OCR without modification (Section 3.2.1).

Architecture. The vision tokenizer combines a SAM-base model (80M parameters) with two subsequent convolutional layers. SAM (Segment Anything Model) provides a pre-trained backbone for extracting rich per-patch features. The convolutional layers then reduce both spatial resolution and channel dimensionality.

Compression ratio. The key property is a 16× token compression relative to naive patch embedding. Specifically, if we think of dividing the input image into a grid of small patches (as ViTs typically do), the SAM-conv pipeline outputs one visual token for every 16×16 region of that grid. This is achieved through window attention within SAM that restricts the receptive field, trading global context within the tokenizer for dramatically reduced output cardinality.

Rationale for 16× compression. The paper states two motivations for retaining this compression (Section 3.2.1):

1. Computational cost for downstream global attention: The LLM-style encoder applies bidirectional attention over all visual tokens. With quadratic complexity in sequence length, reducing tokens by 16× reduces attention computation by 256×. This is what makes packing 500M parameters of Qwen2-0.5B into the encoder feasible without exploding compute.

2. Activation memory: Each transformer layer stores intermediate activations for backpropagation. Reducing token count by 16× proportionally reduces memory, enabling training on 40GB GPUs (as specified in Section 4.2.2).

Parameter count alignment. The paper explicitly notes that the tokenizer's 80M parameters are "comparable to the typical 100M parameters used for text input embeddings in LLMs" (Section 3.2.1). This is a deliberate design symmetry: just as an LLM uses ~100M parameters in its embedding layer to map discrete token IDs to continuous vectors, the vision pipeline uses ~80M parameters to map pixels to compressed visual tokens. The scale is matched, suggesting a unified view of multimodal encoding where different modalities have modality-specific "embedding" modules of comparable size feeding into a shared processing backbone.

Output dimensions. The final convolutional layer's output dimension is reduced from 1024 (in DeepEncoder) to 896 (in DeepEncoder V2). This reduction aligns the dimension with Qwen2-0.5B's native hidden size (896 for the 0.5B variant), allowing the visual tokens to concatenate directly with the LLM-style encoder's token embeddings without an intermediate projection. This is an important practical detail: it avoids inserting a learned linear projection between the tokenizer and the encoder, which would add parameters and potentially introduce a training instability point.

Replaceability. The paper explicitly notes this compression-based tokenizer is "not mandatory and can be replaced with simple patch embedding" (Section 3.2.1). The architectural innovation is in the LLM-style encoder that follows, not in the tokenizer itself. The SAM-conv pipeline is retained for efficiency but is orthogonal to the core contribution.

The LLM-Style Encoder: Why Qwen2-0.5B and the Dual-Stream Attention Mask

This is the paper's central technical contribution—replacing the CLIP ViT with a decoder-only language model repurposed as a vision encoder through a customized attention mask and learnable causal flow queries.

Why a language model architecture? The choice of a decoder-only transformer (Qwen2-0.5B, 500M parameters) over a ViT (CLIP, 300M parameters) is motivated by three considerations (Section 3.2.2):

1. Parameter parity. 500M parameters is comparable to CLIP ViT's 300M, so the replacement does not "introduce excessive computational overhead" (Section 3.2.2). The encoder's compute budget remains roughly in the same regime.

2. Causal attention natively. Decoder-only LLMs are inherently causal—they use triangular attention masks where each token attends only to preceding tokens. This is exactly the mechanism needed to create causal flow among queries. Repurposing an LLM architecture gives you causal attention "for free" as a built-in property, rather than having to retrofit it onto a bidirectional architecture.

3. Future infrastructure compatibility. LLM architectures benefit from massive engineering investment—efficient attention mechanisms (FlashAttention, DeepSeek-V3's MLA), MoE parallelism, sequence packing, and optimized CUDA kernels. By adopting an LLM architecture for the vision encoder, DeepEncoder V2 automatically inherits all these optimizations. This is a strategic choice for scaling: as LLM training infrastructure improves, the vision encoder improves with it without additional engineering effort.

The dual-stream attention mask. The core innovation is not the LLM architecture itself but how it is used. A standard Qwen2-0.5B layer applies causal self-attention over its entire input sequence. DeepEncoder V2 modifies this at the attention mask level: instead of one uniform mask, it constructs a block-structured mask that applies different attention patterns to different token types.

The attention mask, defined in Equation 1, is:

$M = \begin{bmatrix} \mathbf{1}_{m \times m} & \mathbf{0}_{m \times n} \\ \mathbf{1}_{n \times m} & \text{LowerTri}(n) \end{bmatrix}$

where $m$ is the number of visual tokens, $n$ is the number of causal flow queries (with $n = m$), $\mathbf{1}_{a \times b}$ is a matrix of all-ones with dimensions $a \times b$, $\mathbf{0}_{a \times b}$ is a matrix of all-zeros with dimensions $a \times b$, and $\text{LowerTri}(n)$ is an $n \times n$ lower triangular matrix with ones on and below the diagonal and zeros above.

What the mask computes: the attention mask $M_{ij}$ determines whether token $i$ can attend to token $j$ (1 = yes, 0 = no). The mask is applied element-wise to the attention logits before softmax: positions where $M_{ij} = 0$ have their logit set to $-\infty$, producing zero attention weight after softmax. The block structure encodes four distinct attention patterns:

• Top-left block ($\mathbf{1}_{m \times m}$): visual-to-visual attention. All visual tokens attend to all other visual tokens bidirectionally. This preserves CLIP's global receptive field—every visual token can incorporate information from every other visual token, regardless of spatial position. This is what allows the encoder to understand the full image structure.

• Top-right block ($\mathbf{0}_{m \times n}$): query-to-visual attention, prevented. Visual tokens CANNOT attend to causal flow queries. The paper states this explicitly in Section 3.2.2: "visual tokens utilize bidirectional attention" among themselves only, and queries are appended "after visual tokens as a suffix." The zeros ensure visual token representations are not influenced by query content, keeping the visual encoding "pure" and not contaminated by the reordering process. This is crucial: if visual tokens could attend to queries, the queries would be able to modify the visual representation itself, creating a circular dependency where the ordering process changes what is being ordered.

• Bottom-left block ($\mathbf{1}_{n \times m}$): causal-flow-query-to-visual attention. Every causal flow query attends to ALL visual tokens. This gives queries full access to the visual information—each query can "look at" any part of the image, allowing it to select and reorder visual content based on semantic criteria rather than spatial proximity. This is the bridge that allows semantic reordering: queries can attend to visual tokens in any order, building up a causally-ordered representation by choosing which visual information to incorporate at each step.

• Bottom-right block ($\text{LowerTri}(n)$): query-to-query attention. Causal flow queries attend to preceding queries only (including themselves, since the diagonal is 1). Query $k$ attends to queries $0$ through $k$, but NOT to queries $k+1$ through $n-1$. This creates the causal flow: each query's representation is informed by the visual content and by the sequence of preceding queries' representations, enabling cumulative semantic construction where later queries build on the "understanding" distilled by earlier ones.

Why this form over alternatives:

• Why not make visual tokens causal too? If visual tokens used causal attention, they would lose their global receptive field—a visual token on the left side of the image could not attend to a semantically related token on the right side if it appeared later in the sequence. This would severely degrade the encoder's ability to understand image structure. Bidirectional attention for visual tokens is necessary; causal attention for queries is what adds ordering capability on top.

• Why not enable visual-to-query attention (make the top-right block ones)? This would create a feedback loop where queries influence visual tokens, which then influence queries in subsequent layers. The paper does not explicitly discuss this choice, but the architecture suggests a design principle of unidirectional information flow: visual tokens are the "input" that queries process; queries should not modify the visual input they are trying to reorder. Keeping the top-right block zero ensures the visual representation is a stable foundation that queries can selectively attend to.

• Why equal cardinality ($n = m$)? DETR uses 100 queries for object detection regardless of the number of feature tokens; Q-former uses 32 queries regardless of CLIP output count. DeepEncoder V2 explicitly chooses $n = m$—one causal flow query for every visual token. The paper's justification is "sufficient capacity for re-fixation" (Section 3.2.2 and the Introduction's contribution 3). The reasoning: reordering $m$ tokens into a new sequence potentially requires representing each token's new position and semantic context. If there were fewer queries than tokens, information would be lost in the compression—the model would have to decide which visual information to discard at the reordering stage. With equal cardinality, every piece of visual information has a corresponding causal query slot that can carry it into the reordered sequence in its new semantic position. The "re-fixation" analogy is that human vision doesn't discard visual information when re-scanning—it re-examines the same visual field in a different order.

What "causal flow" means operationally. Consider two scenarios:

1. Standard encoder (raster order): Tokens $t_0, t_1, t_2, \dots$ correspond to spatial positions $(0,0), (0,1), (0,2), \dots$. A two-column document has tokens from the left column interleaved with tokens from the right column at every row boundary. The decoder receives this interleaved sequence and must disentangle it.

2. DeepEncoder V2 (causal flow reordering): Query $q_0$ attends to all visual tokens and (having no preceding queries) simply selects which visual information should come first—perhaps the document title. Query $q_1$ attends to all visual tokens AND to $q_0$'s representation, so it can select information that follows naturally from the title—perhaps the abstract text. Query $q_k$ for increasing $k$ progressively builds a semantically ordered sequence, where each step can see what was selected previously and choose what should come next. The final sequence $q_0, q_1, \dots, q_{n-1}$ is a causally-ordered representation of the image where the ordering reflects semantic reading logic rather than spatial coordinates. Only this reordered sequence is fed to the decoder.

The prefix concatenation design. Visual tokens are prepended as a prefix to the causal flow queries in the input sequence. The full input to the encoder is $[\text{VisualToken}_1, \dots, \text{VisualToken}_m, \text{Query}_1, \dots, \text{Query}_n]$. This ordering interacts with the attention mask: the first $m$ positions apply bidirectional attention (left block), while the last $n$ positions apply causal attention with visual access (right blocks). The prefix design keeps visual tokens "active throughout all layers" (Section 3.2.2) because they are present in the key-value cache for every query position. This means each transformer layer has access to the full visual context, not just a compressed bottleneck.

Failed alternative: cross-attention in encoder-decoder. The paper reports (Section 3.2.2) that experiments with "cross-attention in an mBART-style encoder-decoder structure fail to converge." In this alternative, visual tokens would be processed by a separate encoder with self-attention, and queries would be processed by a decoder with cross-attention to the encoder outputs. The hypothesized failure reason is "insufficient visual token interaction when isolated in a separate encoder"—the encoder-decoder separation prevents queries in later decoder layers from attending to visual tokens processed in earlier encoder layers, creating an information bottleneck. In contrast, the prefix design in a single decoder stack means queries at layer $L$ can attend to visual tokens that have themselves been processed through $L$ layers of bidirectional attention, enabling deep, multi-layer integration of visual information into the reordering process.

The Causal Flow Query Mechanisms: Cardinality, Resolution, and Multi-Crop

The causal flow queries are learned embeddings—parameter vectors initialized randomly and updated during training, conceptually similar to DETR's object queries or Q-former's compression queries but serving the distinct function of causal reordering.

Query cardinality calculation. The number of queries $n$ equals the number of visual tokens $m$, which in turn depends on the image resolution input to the encoder:

$n = m = \frac{W \times H}{16^2 \times 16}$

where $W$ and $H$ are the width and height of the image input to the encoder, and $16^2 \times 16 = 4096$ is the compression factor. Specifically: the SAM-conv pipeline compresses by $16\times$ (a factor of 16 in each spatial dimension, so $16^2$ in total area), and the formula includes an additional factor of 16 compression—producing one causal flow query for every 4,096 input pixels.

What the cardinality means concretely: For the standard global view resolution of $1024 \times 1024$, we have $n = m = \frac{1024 \times 1024}{4096} = 256$ queries. For a local crop at $768 \times 768$, we have $n = m = \frac{768 \times 768}{4096} = 144$ queries. The numbers 256 and 144 recur throughout the paper because they are the fixed query counts for the two standardized resolutions.

Why fixed query sets instead of dynamic cardinality? The paper states (Section 3.2.3) that fixed query configurations at predefined resolutions are adopted "to avoid maintaining multiple query sets for different resolutions." If query embeddings were dynamically generated based on image size, each resolution would require its own set of learned parameters or a resolution-adaptive generation mechanism. By standardizing on two resolutions (1024×1024 and 768×768), the model only needs to learn two sets of query embeddings: query_global with 256 embeddings and query_local with 144 embeddings.

The multi-crop strategy. To handle images larger than 1024×1024, the paper employs a multi-crop approach:

• Global view: The entire image is resized to 1024×1024 and processed with 256 causal flow queries. This captures the full-page layout and overall structure.

• Local crops: The image is divided into $k$ crops of size 768×768, where $k \in \{0, 1, \dots, 6\}$. Each crop is processed independently with the same 144 query_local embeddings, and the resulting reordered tokens from all crops are concatenated with the global view tokens. The number of crops is determined by image size: "no cropping is applied when both image dimensions are smaller than 768" (Section 3.2.3), and up to 6 crops for larger images.

• Total output range: The LLM sees between $0 \times 144 + 256 = 256$ tokens (global only, for small images) and $6 \times 144 + 256 = 1120$ tokens (global + 6 crops, for large documents). This range $[256, 1120]$ is deliberately chosen: the lower bound (256) matches DeepSeek-OCR's tokenization of 1024×1024 images, and the upper bound (1120) "matches Gemini-3 Pro's maximum visual token budget" (Section 3.2.3).

Why multi-crop over variable resolution processing? Processing the entire image at a higher resolution would produce more visual tokens, requiring more causal flow queries and increasing the encoder's computational cost quadratically. Multi-crop avoids this: each crop is processed independently at a fixed resolution with a fixed number of queries, so the encoder's per-forward-pass cost is constant regardless of the original image size. Only the number of forward passes varies. This is a practical efficiency tradeoff: crops lose some cross-region context compared to processing the whole page at high resolution, but the global view provides coarse context and the computational savings enable deployment on fixed hardware budgets.

The "re-fixation" capacity argument. The equal cardinality property $n = m$ is specifically motivated as providing "sufficient capacity for re-fixation" (Section 3.2.2 and Introduction). The human vision analogy: when reading a complex document, the eye does not just scan each word once in reading order—it frequently re-fixates on previous elements (re-reading a confusing sentence, checking a figure caption while reading the main text, glancing back at a section header). This re-fixation implies that visual information may appear multiple times in the reordered sequence at different positions, corresponding to different semantic contexts. With $n = m$, each causal query has the capacity to represent a "fixation" that could reference any visual token, potentially allowing the same visual content to be represented at multiple positions in the causal flow (e.g., a figure appearing both when first encountered and again when its caption is processed).

Limitation of equal cardinality. The paper acknowledges in Section 6.1 that equal cardinality may be insufficient: "to enable multiple re-examinations and multi-hop reordering of visual content, we may need substantially longer causal flow tokens than the original visual token sequence." This suggests that $n > m$—more queries than visual tokens—would allow richer reordering with genuine multi-pass re-fixation, where the same visual content is revisited at multiple points in the causal sequence. The current implementation with $n = m$ represents a one-to-one remapping rather than a true multi-hop reasoning process.

Formalizing the Full Pipeline: The Encoder-Decoder Equation

Equation 2 formalizes the entire DeepSeek-OCR 2 forward pass from raw image to output logits:

$\mathbf{O} = \mathcal{D}\left(\pi_Q\left(\mathcal{T}^L\left(\mathcal{E}(\mathbf{I}) \oplus \mathbf{Q}_0; \mathbf{M}\right)\right)\right)$

where $\mathbf{I} \in \mathbb{R}^{H \times W \times 3}$ is the input image with height $H$, width $W$, and 3 color channels; $\mathcal{E}$ is the vision tokenizer (SAM-conv) mapping images to $m$ visual tokens $\mathbf{V} \in \mathbb{R}^{m \times d}$ where $d = 896$ is the token dimension; $\mathbf{Q}_0 \in \mathbb{R}^{n \times d}$ is the matrix of $n = m$ learnable causal query embeddings (initialized randomly); $\oplus$ denotes sequence concatenation producing a combined sequence of length $m + n$ with dimension $d$; $\mathcal{T}^L$ represents $L$ transformer layers (Qwen2-0.5B's full depth) each applying masked self-attention; $\mathbf{M} \in \{0, 1\}^{2n \times 2n}$ is the block causal attention mask from Equation 1 (with $m = n$); $\pi_Q$ is the projection operator that extracts the last $n$ tokens from the encoder output, i.e., $\mathbf{Z} = \mathbf{X}_{m+1:m+n}$ where $\mathbf{X}$ is the layer output; $\mathcal{D}$ is the DeepSeek-MoE language decoder (3B parameters, ~500M active); and $\mathbf{O} \in \mathbb{R}^{n \times |\mathcal{V}|}$ is the output logits over the LLM's vocabulary $\mathcal{V}$ for $n$ decoding steps.

What this computes operationally, step by step:

1. Image to visual tokens ($\mathcal{E}(\mathbf{I})$): The SAM-conv tokenizer takes a $1024 \times 1024 \times 3$ tensor (or $768 \times 768 \times 3$ for crops) and outputs 256 (or 144) feature vectors of dimension 896. Each vector represents a compressed patch of the image with 16×16 compression in each spatial dimension plus the additional factor of 16 from the formula $W \times H / (16^2 \times 16)$.

2. Concatenation with queries ($\mathcal{E}(\mathbf{I}) \oplus \mathbf{Q}_0$): The visual tokens $\mathbf{V}_{m \times d}$ and query embeddings $\mathbf{Q}_0$ (also $n \times d$ with $n = m$) are stacked vertically to form a $2n \times d$ matrix. The first $n$ rows are visual tokens; the last $n$ rows are query tokens.

3. Masked transformer processing ($\mathcal{T}^L$): The combined sequence passes through $L$ transformer layers. In each layer, every token computes self-attention over the sequence, but the attention mask $\mathbf{M}$ restricts which pairs can interact. Visual tokens (positions 0 to $n-1$) compute attention weights over all visual tokens (full bidirectional) but get zero weight for queries. Query tokens (positions $n$ to $2n-1$) compute attention weights over all visual tokens (full access) and over preceding queries only (causal triangular). After $L$ layers of this masked processing, each query token's representation $\mathbf{X}_{m+i}$ encodes: (a) the full visual content of the image (through bidirectional visual-to-visual processing), (b) the specific visual features it chose to attend to (through query-to-visual attention), and (c) the sequence of preceding queries' representations (through causal query-to-query attention).

4. Extraction ($\pi_Q$): Only the query outputs (positions $n$ to $2n-1$) are kept. The visual token representations are discarded. This slicing produces an $n \times d$ matrix $\mathbf{Z}$—the reordered visual representation. Note that $\mathbf{Z}$ has the same dimensions as the visual input $\mathbf{V}$, but the ordering of rows reflects causal semantic flow rather than spatial coordinates.

5. LLM decoding ($\mathcal{D}$): The reordered tokens $\mathbf{Z}$ are prepended to the text prompt embeddings and the DeepSeek-MoE decoder generates output tokens autoregressively. The decoder applies standard causal self-attention over the combined (visual + text) sequence, so the visual tokens' ordering directly influences the decoder's processing.

Why this form:

• $\pi_Q$ only extracts queries: The design discards the original visual token representations. This is intentional—the queries have already distilled and reordered the visual information; passing the original visual tokens to the decoder as well would provide the decoder with both spatial-ordered and semantically-ordered representations, creating ambiguity about which to trust. By providing only the reordered sequence, the decoder is forced to rely on the causal flow ordering that the encoder has learned.

• Cascade of two 1D causal reasoners: The encoder $\mathcal{T}^L$ applies causal attention among queries (through the $\text{LowerTri}(n)$ mask) and the decoder $\mathcal{D}$ applies causal attention over the full sequence. Together, they form two stages of causal processing: the first reorders visual information, the second reasons over the reordered sequence. The paper frames this as the architectural hypothesis of "whether 2D image understanding can be effectively achieved through two-cascaded 1D causal reasoning structures" (Section 1).

• No learned projection between encoder and decoder: The encoder output $\mathbf{Z}$ goes directly to the decoder without an intermediate projector. This contrasts with architectures like BLIP-2 that place a Q-former or linear projection between encoder and decoder. The absence of a projector means the encoder itself is responsible for producing representations that align with the decoder's embedding space—this alignment is learned during the joint training in Stage 2 (Section 4.2.2), where both encoder and decoder are optimized together.

Training Pipeline: Three-Stage Specialization with Progressive Freezing

The paper trains DeepSeek-OCR 2 in three sequential stages, each with different frozen/unfrozen components and different learning objectives. The stages reflect a deliberate progression from foundational encoder capabilities to efficient decoder specialization.

Stage 1: DeepEncoder V2 Pretraining (Section 4.2.1)

• What is trained: The vision tokenizer (SAM-conv) and the LLM-style encoder (Qwen2-0.5B) are trained jointly. The decoder used during this stage is NOT the final DeepSeek-MoE decoder—it is a lightweight OPT-IML model used only as a training auxiliary.

• Objective: Standard language modeling (next token prediction). The encoder processes images, produces reordered visual tokens, and these are fed to the OPT-IML decoder which generates text. The loss is cross-entropy between predicted and ground-truth text tokens.

• Key design choice—auxiliary decoder only: After Stage 1, "only the encoder parameters are retained for subsequent stages" (Section 4.2.1). The OPT-IML decoder is discarded. This is analogous to pretraining a vision encoder with a classification head and then discarding the head—the auxiliary decoder provides a learning signal for the encoder without committing to the final decoder architecture.

• Resolution strategy: Two separate dataloaders at $768 \times 768$ and $1024 \times 1024$ resolutions are used. The paper does not specify whether the model alternates between resolutions per batch or uses separate phases, but the existence of two dataloaders implies the encoder learns to handle both resolution-specific query sets (query_local and query_global).

• Initialization: The vision tokenizer is initialized from DeepEncoder's pretrained SAM-conv weights (not from scratch), providing a warm start for image compression. The LLM-style encoder is initialized from Qwen2-0.5B-base weights (a general-purpose language model), not from a vision-pretrained model. This is critical: the encoder starts from a text-only LLM initialization and must learn to process visual tokens during Stage 1.

• Hyperparameters: AdamW optimizer with cosine learning rate decay from $1 \times 10^{-4}$ to $1 \times 10^{-6}$. Training on 160 A100 GPUs (20 nodes × 8 GPUs) with batch size 640 for 40,000 iterations. Sequence packing at 8K length is used (multiple image-text pairs packed into each 8K-token training sequence to maximize GPU utilization). Approximately 100 million image-text pair samples are consumed (640 batch × 40k iterations = 25.6M sequences, but sequence packing increases the effective sample count).

• What the encoder learns in Stage 1: The encoder must simultaneously learn three capabilities: (a) to process visual tokens effectively (the LLM backbone has never seen visual inputs), (b) to use the causal flow queries to produce semantically meaningful reorderings, and (c) to produce representations that enable the auxiliary decoder to generate accurate text. The language modeling objective provides a strong signal: if the encoder's reordering is poor, the decoder will produce garbled text, and gradients will flow back through the decoder to update the encoder.

Stage 2: Query Enhancement (Section 4.2.2)

• What is trained: The vision tokenizer (SAM-conv) is now FROZEN. The LLM-style encoder and the final DeepSeek-MoE decoder are trained jointly. This is the stage where encoder and decoder learn to work together.

• Objective: Same language modeling objective, but now with the final 3B MoE decoder instead of the lightweight auxiliary. The encoder's query representations are "enhanced" to produce tokens that are optimally informative for the specific decoder architecture.

• Multi-crop strategy activated: Unlike Stage 1's separate resolutions, Stage 2 unifies them "into a single dataloader via multi-crop strategy" (Section 4.2.2). Each training image produces both a global view (1024×1024) and possibly local crops (768×768), with all views processed in a single forward pass (or across multiple micro-batches).

• Pipeline parallelism configuration: The model is distributed across 4 pipeline stages (PP0–PP3): PP0 hosts the vision tokenizer, PP1 hosts the LLM-style encoder, and PP2–PP3 host the DeepSeek-MoE decoder (6 layers per stage). With 160 GPUs (40GB per GPU), the configuration is 40 data parallel replicas × 4 GPUs per replica (one GPU per pipeline stage). Global batch size is 1280, optimized with AdamW and cosine learning rate decay from $5 \times 10^{-5}$ to $1 \times 10^{-6}$ over 15,000 iterations.

• Why freeze the tokenizer at this stage: The vision tokenizer is the most mature component (initialized from DeepEncoder), and its output quality directly affects both encoder and decoder training stability. Freezing it reduces the optimization burden and prevents the encoder and decoder from "blaming" tokenizer changes for their own errors. It also speeds up training by eliminating gradients for the 80M tokenizer parameters.

• What changes in the encoder during Stage 2: With the final decoder providing a stronger and more architecture-specific learning signal than the Stage 1 auxiliary, the encoder's query representations are refined to be optimal specifically for the DeepSeek-MoE decoder's attention patterns, MoE routing, and capacity. The queries learn to produce not just semantically ordered tokens but tokens that the decoder's experts can efficiently process.

Stage 3: Decoder Specialization (Section 4.2.3)

• What is trained: ALL encoder parameters (vision tokenizer + LLM-style encoder) are frozen. Only the DeepSeek-MoE decoder parameters are updated.

• Objective: Language modeling, with the encoder providing fixed, high-quality reordered visual tokens.

• Learning rate: Starting from the Stage 2 final learning rate, another cosine decay from $1 \times 10^{-6}$ to $5 \times 10^{-8}$ over 20,000 iterations. The extremely low learning rates reflect fine-tuning of an already well-trained decoder.

• Why freeze the encoder at this stage: Two practical reasons: (1) Training speed—"this stage accelerates training (more than doubles the training speed under the same global batch size)" (Section 4.2.3) because gradients do not need to be computed or communicated for the encoder's 500M+80M parameters. (2) Stability—the encoder's ordering has been optimized in Stage 2; further changes would force the decoder to continuously adapt to changing visual representations, potentially slowing convergence. Freezing the encoder allows the decoder to "settle" and specialize to the specific ordering the encoder produces.

• What the decoder learns in Stage 3: With a fixed visual input distribution, the decoder can specialize its attention patterns and expert routing to the specific characteristics of the reordered visual tokens. This is analogous to continued pretraining of an LLM on a fixed data distribution—the model can "compress" its knowledge of the visual encoding into more efficient parameter configurations.

• Data consumption rate: The paper highlights that Stage 3 is designed to "rapidly consume training data" (Section 4.2.3). The combination of frozen encoder (eliminating backward passes through half the model) and dedicated decoder optimization enables higher throughput, allowing more training data to be processed in the same wall-clock time.

Training Data and Balancing

The training data engine is "the same data sources as DeepSeek-OCR" (Section 4.1) with two modifications:

• OCR 1.0 rebalancing: Data is partitioned by content type into text, formulas, and tables with a 3:1:1 sampling ratio. This gives text content 3× the sampling weight of formulas or tables, reflecting both the natural distribution of document content and the greater difficulty of text recognition (which has more varied fonts, layouts, and languages). The 80% overall OCR mixture means these three categories collectively constitute 80% of training samples, with the remaining 20% being general vision data.

• Layout label refinement: Semantically similar layout categories are merged, e.g., "figure caption" and "figure title" are unified into a single label. This reduces label sparsity and prevents the model from learning artificial distinctions between subtypes that share the same visual function.

The paper explicitly states these are "minimal differences" (Section 4.1) from DeepSeek-OCR, allowing performance improvements to be attributed to the architectural change rather than data changes.

Design Choices and Alternatives Considered

Why replace CLIP with an LLM rather than modifying CLIP? The paper doesn't entertain modifying CLIP to add causal queries (e.g., appending causal query tokens to a ViT) because CLIP's architecture is fundamentally bidirectional—its attention head implementation, positional encoding scheme, and training objective are all designed for non-causal processing. Retro-fitting causal attention into CLIP would require modifying the core attention mechanism, potentially breaking its pretrained visual understanding. Starting from a decoder-only LLM with native causal attention gives a clean architectural basis for the dual-stream mask without legacy constraints.

Why Qwen2-0.5B specifically? The paper mentions parameter parity with CLIP ViT (300M vs. 500M) as a consideration but doesn't provide a detailed justification for the Qwen2 choice over other small LLMs (e.g., LLaMA-500M, OPT-350M). The likely factors: Qwen2 is a well-documented, openly available model with strong pretraining; 0.5B sits in a "sweet spot" where it's large enough to learn meaningful visual representations but small enough to keep encoder cost comparable to CLIP; and the 896 hidden dimension aligns with the tokenizer output without requiring an intermediate projection.

Why the mBART-style alternative failed. The paper's report that encoder-decoder cross-attention "fail[s] to converge" (Section 3.2.2) is an important negative result. The hypothesized reason—"insufficient visual token interaction when isolated in a separate encoder"—suggests that deep, multi-layer interaction between visual tokens and queries is essential. In an encoder-decoder design, the encoder processes visual tokens to a fixed representation, and the decoder accesses this representation through cross-attention. But the visual tokens can only interact with each other within the encoder; they cannot interact with the queries that are trying to reorder them. The prefix design, by contrast, allows visual tokens and queries to co-process within the same transformer stack, enabling mutual refinement layer by layer. This finding has implications beyond this paper: it suggests that effective multimodal processing in transformer architectures benefits from co-locating different modalities in the same attention layers rather than separating them into encoder-decoder stacks.

Why equal cardinality ($n = m$) over compression (like Q-former's 32 queries)? The paper discusses two competing goals for the queries: (1) compression—producing fewer tokens than the visual input to reduce decoder cost, and (2) reordering—producing a semantic sequence that preserves visual information. Q-former optimizes for (1) at the cost of (2): with only 32 queries, information is inevitably lost. DeepEncoder V2 explicitly optimizes for (2): by keeping $n = m$, no information is lost during reordering. Compression is handled separately by the vision tokenizer's 16× reduction, so the queries don't need to compress—they only need to reorder. This decomposition of concerns (tokenizer handles compression, encoder handles ordering) is cleaner than Q-former's approach of having queries do both simultaneously.

Why feed only causal flow tokens to the decoder (not visual tokens)? If both the original visual tokens and the causal flow tokens were fed to the decoder, the model would have two representations of the same visual content—one in spatial order, one in causal-semantic order. The decoder would need to learn to either (a) ignore one representation, (b) integrate both, or (c) dynamically select between them per token. Any of these increases the decoder's cognitive load. Providing only the reordered query tokens forces the decoder to trust the encoder's reordering, creating a clear division of responsibility: encoder handles visual understanding and ordering, decoder handles linguistic reasoning over the ordered content.

Summary of Design Principles

The architecture embodies several principled design decisions:

1. Concern decomposition: Compression (vision tokenizer) is separated from ordering (causal flow queries) is separated from linguistic reasoning (LLM decoder). Each component has a single, well-defined responsibility.

2. Architectural parity with LLMs: By using an LLM-style encoder, the visual pipeline inherits all optimizations from the language modeling community without additional engineering effort.

3. Causal reordering as a first-class operation: Rather than having the decoder implicitly reconstruct ordering from spatial tokens, the encoder explicitly performs reordering through learned causal queries. This makes the ordering process visible, trainable, and interpretable.

4. Unidirectional information flow from visual to causal: Visual tokens can influence queries, but queries cannot influence visual tokens (top-right block of the mask is zero). This prevents circular dependencies and ensures the visual representation is a stable foundation.

5. Progressive freezing for training stability: The three-stage pipeline freezes components from most fundamental (tokenizer) to least fundamental (decoder), ensuring that foundational representations stabilize before higher-level components specialize to them.

4. Key Insights and Innovations

Innovation 1: Causal Reordering as a First-Class Architectural Primitive for Vision Encoding

The dominant assumption in vision-language models is that visual tokens reach the decoder arranged in raster-scan order, and the decoder's job is to reconstruct semantic relationships from this spatially-imposed sequence. Every major VLM architecture—from LLaVA to Qwen-VL to InternVL—inherits this ordering without questioning it. The encoder's job is to extract features; the decoder's job is to make sense of whatever order those features happen to arrive in.

DeepEncoder V2 fundamentally reframes this division of labor. It introduces causal reordering as an explicit, learned operation that sits between feature extraction and linguistic reasoning. The encoder no longer just extracts features—it now has a second, equally important responsibility: determining in what order those features should be presented for downstream reasoning. This is not a minor tweak to the attention pattern. It's a conceptual shift in what a vision encoder is: from a feature extractor that produces content, to a visual reasoning module that produces both content and a semantically meaningful sequence over that content.

The shift is best understood by comparing against what bidirectional attention alone achieves. In a standard ViT encoder, bidirectional self-attention computes rich token-level representations that capture cross-token relationships—a token representing a figure caption can incorporate information from the figure itself, a table cell can attend to its column header, and so on. The content is there. But the arrangement of this content in the output sequence is determined entirely by the initial spatial coordinates determined at patch embedding time, which the attention mechanism never changes. The encoder sees everything but says nothing about what order everything should be processed in.

What makes DeepEncoder V2's contribution genuinely novel—rather than an obvious extension of bidirectional attention—is that it recognizes permutation equivariance as the root cause of the ordering problem and addresses it through asymmetry in the attention mask. Bidirectional attention is permutation-equivariant: shuffling the input tokens produces a correspondingly shuffled output, with each token's content unchanged. Causal attention breaks this symmetry—the ordering of queries matters, because query $\,k$ can attend to queries $\,0$ through $\,k-1$ but not $\,k+1$ through $\,n-1$. This asymmetry forces the model to learn an ordering that matters for the downstream task, since the causal structure means that which information appears early in the query sequence constrains what can appear later. The model cannot simply reproduce visual tokens in some arbitrary permutation—it must learn an ordering where prefixes are maximally informative for predicting suffixes, because that's what the language modeling objective rewards.

This is a fundamental conceptual move, not an incremental refinement. Prior query-based approaches (DETR, Q-former) used parallelized queries with bidirectional self-attention—every query attends to every other query simultaneously, so there's no ordering, no sequential dependency, and no notion of "what should come first." DeepEncoder V2's causal self-attention among queries creates a sequential dependency structure where none existed before. The queries are no longer just a set; they're a sequence with causal flow. This transforms the query mechanism from a compression tool (Q-former) or a detection tool (DETR) into a reasoning mechanism—the queries collectively perform a sequential computation whose output order carries semantic meaning.

The reading-order results in Table 3 provide the cleanest evidence that this reordering is learned, not accidental. Across all nine document types, DeepEncoder V2 produces lower reading-order edit distances than DeepSeek-OCR (e.g., 0.025 vs. 0.052 for PPT, 0.013 vs. 0.021 for Academic Paper, 0.176 vs. 0.217 for Magazine). The consistency of the improvement—every category improves, despite substantial differences in layout complexity across categories—suggests the model has learned a generalizable reordering capability, not just memorized document-type-specific patterns. If the improvement were driven primarily by better feature extraction (the standard CLIP→LLM capacity upgrade), we would expect more mixed results across categories, with some benefiting and others not. Universal reading-order improvement points to a learned sequential reasoning capability.

Innovation 2: Decomposition of 2D Understanding into Two Cascaded 1D Causal Reasoners as an Architectural Hypothesis

The paper's most ambitious conceptual contribution is the architectural hypothesis that 2D visual understanding can be effectively achieved through two cascaded stages of 1D causal reasoning. This is not presented as a proven theorem—the paper is explicit that "achieving this goal remains a long journey" (Section 6.1)—but as a research direction validated through preliminary evidence on document reading.

The hypothesis decomposes 2D understanding into two orthogonal 1D reasoning subtasks:

Stage 1 (encoder): Reading logic reasoning. The encoder takes spatially-structured visual information and performs causal reordering—determining the sequence in which visual content should be examined based on its semantic structure. This is a 1D causal process because each query's representation depends causally on preceding queries' representations, building up a sequential understanding of visual layout. The encoder doesn't need to understand the linguistic content of text (the decoder handles that); it needs to understand visual structure—columns, tables, reading flow, figure-caption relationships—well enough to produce a meaningful order.

Stage 2 (decoder): Visual task reasoning. The decoder takes the causally-ordered visual sequence and performs autoregressive linguistic reasoning—extracting text, solving formulas, recognizing table structures, answering questions. This is standard 1D causal reasoning as performed by any LLM, but critically, it operates over a sequence whose ordering has been semantically optimized rather than spatially determined.

What makes this decomposition intellectually significant is that it addresses a fundamental representational mismatch in VLM architectures without requiring a fundamentally new type of attention or a 2D-native architecture. The hypothesis is: 2D reasoning may not require 2D attention; it may be achievable through proper decomposition into 1D causal subtasks with the right interface between them. This is a genuinely novel framing that suggests a different scaling path for multimodal models—rather than developing increasingly sophisticated 2D attention mechanisms (axial attention, windowed attention with 2D relative position biases, etc.), invest in better 1D causal reordering mechanisms that can translate spatial structure into causal structure.

The evidence supporting this hypothesis comes from the contrast between the encoder and decoder roles. Table 2 shows that DeepEncoder V2 achieves lower edit distances specifically on formula (0.198 vs. 0.236 for DeepSeek-OCR) and table (0.096 vs. 0.123) reconstruction—both tasks that require understanding 2D spatial relationships (superscripts, fraction bars, matrix layouts in formulas; row-column alignment, spanning cells in tables). Yet the encoder itself is a 1D causal architecture. The improvement must come from better reordering—the encoder has learned to linearize 2D formula and table structures into a 1D sequence that the decoder can more easily process, effectively offloading spatial reasoning from the decoder (which must simultaneously handle linguistic generation) to the encoder (which specializes in visual structure).

The paper's honest acknowledgment of current limitations—"to enable multiple re-examinations and multi-hop reordering of visual content, we may need substantially longer causal flow tokens than the original visual token sequence" (Section 6.1)—strengthens rather than weakens this innovation. It identifies a clear next step ($n > m$ for multi-pass reordering) that follows logically from the two-stage framework, suggesting the hypothesis has generative power beyond the current implementation.

Innovation 3: The Dual-Stream Attention Mask as a General Architectural Pattern for Integrating Content Processing with Sequential Reasoning

While the dual-stream attention mask is technically a mechanism (detailed in Section 3.4), its conceptual significance extends beyond the specific implementation in DeepEncoder V2. The paper introduces what amounts to a general design pattern for transformer architectures: concatenate two token types along the sequence dimension, apply bidirectional attention to one type (content tokens that need global processing), apply causal attention to the other type (reasoning tokens that need sequential structure), and allow asymmetric cross-attention (reasoning tokens can attend to content tokens, but not vice versa).

This pattern is significant because it cleanly separates two concerns that are typically conflated in vision-language architectures:

• Content understanding (bidirectional visual attention): What is in the image? What is the relationship between different visual elements? This requires global, all-to-all processing—a figure caption needs to attend to the figure, a table cell needs to attend to its row and column headers, footnotes need to attend to their reference points. Bidirectional attention handles this naturally.

• Sequential reasoning (causal query attention): In what order should this content be processed? What comes first, second, third? This requires sequential dependency—later processing steps should build on earlier ones. Causal attention handles this naturally.

Prior architectures either impose ordering without reasoning (raster-scan, which determines order by spatial coordinates without any learned content processing) or avoid ordering entirely (bidirectional query sets, which process all queries in parallel without any sequential dependency). The dual-stream mask achieves both simultaneously: the visual tokens handle content understanding through bidirectional attention, while the query tokens handle sequential reasoning through causal attention, and the content tokens provide a stable, unordered foundation that the reasoning tokens can selectively attend to.

The asymmetry in the mask—queries can attend to visual tokens, but visual tokens cannot attend to queries—is a non-obvious design choice that has significant conceptual implications. It enforces unidirectional information flow from content to reasoning, preventing the reasoning process from modifying the content representation. This means the visual tokens serve as a "ground truth" representation of the image that the queries can interpret and reorder but cannot alter. If visual tokens could attend to queries, the content representation would become entangled with the ordering process—a circular dependency where what you're ordering depends on how you're ordering it. The asymmetry keeps the content clean and the reasoning accountable: the queries must work with the visual features as they are, not as they would like them to be.

This design pattern has implications beyond document OCR. Any task that involves processing unordered content into an ordered sequence could benefit from this separation: video understanding (frames as content tokens, temporal reasoning as queries), multi-document summarization (documents as content tokens, summary structure as queries), or program synthesis from natural language (specification as content tokens, generated code as queries). The pattern provides a template for how to inject causal reasoning into content-processing pipelines without breaking the content processing's global receptive field.

Innovation 4: Empirical Validation that Semantic Reordering Compensates for Token Compression

The paper's practical finding—91.09% accuracy with only 1,120 visual tokens versus models using 6,000–7,000 tokens (Table 1)—reveals something conceptually important that goes beyond strong benchmark numbers: token count and token ordering are partially substitutable resources. Better ordering allows the decoder to extract more information from fewer tokens, because each token arrives at the right time in the causal sequence and in the right semantic context.

This substitutability is not obvious a priori. One could imagine that compression loss is fundamental—information discarded by the tokenizer cannot be recovered downstream, regardless of how cleverly the remaining tokens are arranged. Under this view, more tokens always means more information, and the only question is whether the additional information is worth the computational cost. DeepEncoder V2's results suggest a different dynamic: when tokens are poorly ordered (raster scan), the decoder wastes capacity disentangling spatial artifacts from semantic structure, so even a large token budget under-delivers. When tokens are well-ordered (causal flow), the decoder can extract meaning efficiently, and a smaller token budget over-delivers relative to its information content.

Table 1 provides strong evidence for this substitutability. DeepSeek-OCR 2 at 1,120 tokens outperforms InternVL3 at 7,000+ tokens (91.09% vs. 80.33%), Qwen2.5-VL-72B at 6,000+ tokens (91.09% vs. 87.02%), and Gemini-2.5 Pro at its own budget ceiling (91.09% vs. 88.03% at 1,120 tokens for Gemini). The comparison against models with 5–7× more visual tokens is striking—if token count were the primary driver of performance, these models should dominate. That a 1,120-token model surpasses them implies that ordering efficiency provides a larger effective information gain than what's lost through compression. Put differently: 1,120 well-ordered tokens contain more usable information for the decoder than 7,000 poorly-ordered tokens, even if the 7,000-token representation contains more raw information in an information-theoretic sense.

The practical significance of this finding is substantial. Visual token count is the primary cost driver in vision-language model inference—each additional token adds quadratic attention cost in the decoder. If ordering improvements can achieve the same accuracy with 5–7× fewer tokens, the computational savings at deployment scale are dramatic. This reframes the research question around visual token budgets: rather than "how many tokens do we need to capture all visual information?", the question becomes "how can we order tokens to maximize extractable information per token?" The answer may involve architectural investments in reordering that pay for themselves many times over through reduced decoder costs.

The finding also has implications for scaling. If ordering and token count are substitutable, then improving ordering quality becomes a scaling axis parallel to increasing token count. Future work could investigate "ordering scaling laws": how does performance improve as we invest more compute in the encoder's reordering capability versus investing in more decoder tokens? The paper doesn't explore this, but the conceptual framework it establishes makes the question natural to ask.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The primary benchmark is OmniDocBench v1.5 (Ouyang et al., 2025), comprising 1,355 document pages across 9 categories (magazines, academic papers, research reports, PPT, textbooks, exam papers, newspapers, notes, and colorful textbooks) in both Chinese and English. The paper states this benchmark provides "diverse test samples and robust evaluation criteria" (Section 5). For production evaluation (Section 5.3), the paper uses two additional unlabeled datasets: online-user logs (images uploaded by users of DeepSeek's OCR service) and a pretraining data pipeline (PDF documents processed for LLM training data).

• Base model(s). The system under evaluation is DeepSeek-OCR 2, which consists of: (1) a vision tokenizer (SAM-base, ~80M parameters, initialized from DeepSeek-OCR), (2) DeepEncoder V2 (Qwen2-0.5B LLM-style encoder, ~500M parameters), and (3) a DeepSeek-MoE decoder (3B total parameters, ~500M active parameters, inherited unchanged from DeepSeek-OCR). The primary comparison baseline is DeepSeek-OCR (the predecessor model using CLIP-based DeepEncoder with an otherwise identical decoder and training data engine). The paper also compares against a range of pipeline-based and end-to-end models as reported in the OmniDocBench repository.

• Metrics. The paper uses several evaluation metrics from OmniDocBench v1.5:

  • Overall accuracy (%) — the primary aggregate metric combining all sub-tasks.
  • Text Edit Distance (Text ED) — character-level edit distance for recognized text, where lower is better.
  • Formula CDM (Chemical-structure Deformation Metric) — a metric for formula recognition accuracy (higher is better). The paper does not elaborate on the CDM computation details, treating it as a standard OmniDocBench metric.
  • Table TEDs (Tree Edit Distance similarity) — similarity score for table structure recognition (higher is better).
  • Table TEDs_s — a variant of TEDs specific to table structure.
  • Reading Order Edit Distance (R-order ED) — edit distance measuring correctness of the predicted reading order sequence (lower is better). This metric is highlighted as directly evaluating the causal flow mechanism's primary contribution.
  • Overall Edit Distance — aggregate edit distance across all document elements (used in Table 2).
  • For production evaluation (Section 5.3): Repetition rate (%), defined as the percentage of outputs containing repetitive or semantically garbled text, measured on unlabeled production data where ground truth is unavailable.

• Baselines. The paper evaluates against two categories:

  • Pipeline methods (Table 1, upper section): Marker-1.8.2, MinerU2-pp, Dolphin/Dolphin-1.5, PP-StructureV3, MonkeyOCR-pro-1.2B/3B, MinerU2.5, and PaddleOCR-VL. These are non-end-to-end systems that decompose OCR into separate detection, recognition, and parsing stages. Results are sourced from the OmniDocBench repository.
  • End-to-end models (Table 1, lower section): OCRFlux, GPT-4o, InternVL3, POINTS-Reader, olmOCR, InternVL3.5-241B, MinerU2-VLM, Nanonets-OCR-s, Qwen2.5-VL-72B, Gemini-2.5 Pro, dots.ocr, OCRVerse, Qwen3-VL-235B, and DeepSeek-OCR. Most of these use substantially larger visual token budgets (6,000–7,000+ tokens) than DeepSeek-OCR 2 (1,120 maximum).
  • The primary controlled comparison is against DeepSeek-OCR at 1,156 tokens (9-crops mode), which shares the same training data engine, decoder architecture, and training pipeline, isolating the effect of the encoder architecture change.

• Generation budget / compute accounting. Visual token budget is measured as V-token max — the maximum number of visual tokens fed to the LLM decoder per page. This is the primary resource metric because each visual token incurs quadratic attention cost in the decoder. DeepSeek-OCR 2 constrains its budget to 256–1,120 tokens (global view at 256 tokens + up to 6 local crops at 144 tokens each), with the upper bound matching Gemini-3 Pro's budget. This is compared against DeepSeek-OCR's 1,156 tokens (9-crops) and other models' budgets ranging from ~6,000 to over 7,000 tokens. The paper does not report total FLOPs or wall-clock time for inference.

• Cross-validation / statistical protocol. The paper does not describe any cross-validation or statistical significance testing for the main benchmark results. The OmniDocBench v1.5 results appear to be single-run evaluations on the full 1,355-page test set. There is no mention of multiple evaluation runs, confidence intervals, or statistical tests comparing DeepSeek-OCR 2 against baselines. For the production evaluation, metrics are computed on the full production datasets without statistical testing.

Main Quantitative Results

Overall Benchmark Performance

The headline result appears in Table 1: DeepSeek-OCR 2 achieves 91.09% overall accuracy on OmniDocBench v1.5, using a maximum of 1,120 visual tokens. This represents a 3.73 percentage point absolute improvement over DeepSeek-OCR (87.36% at 1,156 tokens) and establishes the model as the top-performing system on this benchmark among all compared methods.

The per-metric breakdown from Table 1 shows consistent improvements across every sub-metric relative to DeepSeek-OCR:

The largest relative improvement is in formula recognition (+6.17 CDM), consistent with the paper's claim that causal reordering helps linearize 2D structures (formulas have complex spatial layouts with superscripts, subscripts, and fraction bars). The reading order edit distance improvement (−0.028, a 33% relative reduction) directly validates the causal flow mechanism's core purpose—semantically reordering visual information before decoder processing.

Comparison against models with larger token budgets. DeepSeek-OCR 2 at 1,120 tokens outperforms all end-to-end models in Table 1 despite their substantially larger budgets, including:

• InternVL3 (>7,000 tokens): 80.33% vs. 91.09% (+10.76 advantage)
• Qwen2.5-VL-72B (>6,000 tokens): 87.02% vs. 91.09% (+4.07 advantage)
• Gemini-2.5 Pro (1,120 tokens, matched budget): 88.03% vs. 91.09% (+3.06 advantage at equal token count)
• Qwen3-VL-235B (>6,000 tokens): 89.15% vs. 91.09% (+1.94 advantage)
• dots.ocr (>6,000 tokens): 88.41% vs. 91.09% (+2.68 advantage)

The comparison against Gemini-2.5 Pro at equal token budget (1,120) is the most controlled cross-model comparison, isolating the effect of architecture and training rather than token count. The 3.06 percentage point advantage at equal budget supports the claim that causal reordering makes more efficient use of the available visual tokens.

Pipeline model comparison. DeepSeek-OCR 2 also outperforms the best pipeline methods, including PaddleOCR-VL (92.86% vs. 91.09%, −1.77 deficit) and MinerU2.5 (90.67% vs. 91.09%, +0.42 advantage). Notably, PaddleOCR-VL achieves a higher score (92.86%) but is a pipeline system—not an end-to-end model—so the comparison is across architectural categories. Among end-to-end models, DeepSeek-OCR 2 is the clear leader.

Detailed Document-Type Analysis

Table 3 breaks down Text ED and R-order ED across 9 document types for DeepSeek-OCR versus DeepSeek-OCR 2, revealing where improvements concentrate and where weaknesses remain.

Reading order improvements (R-order ED, lower is better). DeepSeek-OCR 2 improves reading order across all nine document types without exception:

The universal improvement pattern is the strongest evidence that the causal flow mechanism generalizes across document layouts rather than memorizing type-specific patterns. Academic papers and research reports—which typically have simpler, more linear layouts—show small improvements (0.008 and 0.005 respectively) because the baseline raster-scan ordering was already reasonably close to the natural reading order. Colorful textbooks (complex multi-element layouts) and magazines (multi-column, irregular layouts) show much larger improvements (−0.059 and −0.041 respectively), precisely where the gap between spatial ordering and semantic reading order is widest.

The Magazine category shows the smallest improvement (−0.001), with DeepSeek-OCR 2 still at a relatively high 0.100 R-order ED. The paper attributes this to two factors (Section 5.2): (1) the lower visual token budget ceiling (1,120 vs. 1,156) may be insufficient for text-super-rich newspaper layouts, and (2) insufficient training data for newspaper-type documents ("only 250k relevant samples"). The Magazine result (also 0.100) suggests a similar limitation

Text recognition (Text ED, lower is better). The text recognition results are more mixed:

Two categories—Book (0.022 → 0.033, +0.011) and Newspaper (0.131 → 0.139, +0.008)—show text recognition degradation. The paper discusses the Newspaper degradation explicitly (Section 5.2), attributing it to the lower visual token budget ceiling and insufficient training data. The Book degradation is not discussed; it is listed in Table 3 without commentary. The contrast is notable because Book reading order improved (−0.013) while text recognition degraded (+0.011), suggesting that for this document type, the reordering mechanism may have traded off some content fidelity for better sequential structure. The paper does not explore this tradeoff.

Formula and table edit distances (Table 2). The more granular breakdown in Table 2 shows per-element-type edit distances:

DeepSeek-OCR 2 achieves lower overall ED (0.100) than Gemini-3 Pro (0.115) at the same maximum token budget, and lower than Seed-1.8 (0.106) despite Seed-1.8 using 4.6× more tokens. The most striking improvement is in formula ED (0.236 → 0.198, a 16% relative reduction), consistent with the hypothesis that causal reordering helps linearize 2D formula structures for the 1D decoder. Table ED also shows substantial improvement (0.123 → 0.096, a 22% relative reduction), suggesting that the reordering mechanism effectively captures table row-column structure.

Comparability note on Table 2. Gemini-3 Pro and Seed-1.8 do not report per-element-type breakdowns (only overall ED), so per-element comparisons are limited to DeepSeek-OCR vs. DeepSeek-OCR 2. The paper sources Gemini-3 Pro and Seed-1.8 numbers from an external repository, not from its own evaluation infrastructure.

Production Readiness: Repetition Rate Reduction

Table 4 reports repetition rate—the primary quality metric for OCR systems in production where ground truth is unavailable:

On online user-uploaded images, DeepSeek-OCR 2 reduces repetition rate from 6.25% to 4.17%—a 2.08 percentage point absolute reduction (33.3% relative reduction). On PDF data processed for LLM pretraining, the improvement is smaller: 0.81 percentage point reduction (3.69% → 2.88%).

The paper interprets the repetition rate reduction as validating "the effectiveness of the DeepSeek-OCR 2 architecture, particularly its logical visual comprehension capabilities" (Section 5.3). The reasoning: repetition in OCR output often stems from garbled reading order—the decoder loses track of where it is in the document and starts repeating content—so lower repetition rates indicate better visual ordering. The larger improvement on online-user-logs (2.08 percentage points) versus PDF data (0.81) suggests that user-uploaded images are more diverse and challenging than standardized PDF documents, providing greater opportunity for the reordering mechanism to make a difference.

Important caveat on production metrics. The repetition rate is measured on unlabeled data where ground truth text is unknown. This means the metric captures only one specific failure mode (repetitive output) and cannot detect other errors (incorrect but non-repetitive text, missing content, hallucinated content). A model could achieve low repetition by being silent (producing minimal output) rather than by being accurate. The paper does not report production accuracy metrics (which would require ground truth) or other quality proxies (output length distribution, vocabulary diversity, etc.), so the repetition rate should be interpreted as a narrow signal of architectural robustness rather than a comprehensive quality measure.

Improvement Headroom Analysis

Table 3 reveals that even with the significant gains, DeepSeek-OCR 2 has clear headroom for improvement. The paper identifies two categories of limitation (Section 5.2):

1. Newspaper and Magazine text recognition: Newspaper Text ED of 0.139 is the highest among all categories and represents a degradation from DeepSeek-OCR's 0.131. Magazine Text ED improves (0.049 → 0.026) but R-order ED shows minimal gain (0.101 → 0.100). The paper attributes newspaper weakness to insufficient training data ("only 250k relevant samples") and notes the lower visual token ceiling may "affect the recognition of text-super-rich newspapers."

2. Remaining reading order errors in complex layouts: Newspaper R-order ED at 0.176 and Magazine at 0.100 are still relatively high compared to simpler documents (Academic Paper at 0.013, Research Report at 0.011). This suggests the causal reordering mechanism still struggles with the most complex, irregular layouts even though it substantially improves over the baseline.

The paper frames these as addressable limitations: newspaper performance can be improved by "increasing the number of local crops" (to raise the visual token budget) or adding more training data, rather than indicating a fundamental flaw in the causal flow approach.

Ablation Studies and Robustness Checks

The paper provides very limited formal ablation studies. This is a significant weakness of the experimental evaluation. The following represent what the paper does report, some of which are implicit ablations rather than controlled experiments:

Architecture upgrade ablation (implicit): DeepSeek-OCR vs. DeepSeek-OCR 2 overall comparison. Table 1 and Table 2 provide the primary ablation of the architectural change—replacing CLIP-based DeepEncoder with LLM-style DeepEncoder V2 while holding training data, decoder architecture, and training pipeline constant. The 3.73% overall improvement and consistent gains across all reading-order metrics represent the aggregate effect of all DeepEncoder V2's innovations (dual-stream attention, causal queries, LLM backbone). However, this comparison conflates multiple changes: the backbone architecture (CLIP ViT → Qwen2-0.5B), the attention mechanism (bidirectional-only → dual-stream), the query mechanism (none → causal flow queries with equal cardinality), and the parameter count (300M → 500M). The paper does not isolate which of these changes contributes how much to the improvement.

Multi-crop token budget ablation (implicit): 1,120 vs. >6,000 tokens across models. The comparison against models using 5–7× more visual tokens (InternVL3, Qwen2.5-VL, etc. in Table 1) implicitly ablates the effect of token budget. The fact that DeepSeek-OCR 2 outperforms these models despite having far fewer tokens supports the claim that ordering quality can substitute for token quantity. However, this is a cross-model comparison confounded by differences in pretraining data, model scale, training objectives, and architecture—not a controlled ablation within a single model family at different token budgets.

Production metric ablation: repetition rate. Table 4 compares DeepSeek-OCR versus DeepSeek-OCR 2 on the same production datasets using the same metric, providing a real-world robustness check of the architectural upgrade. The consistent reduction in repetition rate across two different data distributions (online user images and PDF documents) supports the claim that the improvement is not benchmark-specific.

What the paper does NOT ablate:

• Causal vs. bidirectional queries: The paper does not compare DeepEncoder V2 against a variant where queries use bidirectional self-attention (like Q-former) instead of causal attention. This is the single most important missing ablation, because it would directly test whether causal structure specifically—or the introduction of learnable queries generally—drives the improvement.

• Backbone architecture effect: The paper does not test whether the improvement comes from replacing CLIP ViT with a larger LLM backbone (500M vs. 300M parameters, different pretraining) or from the dual-stream attention mechanism specifically. A CLIP ViT with appended causal queries would isolate the mechanism from the backbone change.

• Equal cardinality necessity: The paper does not test with fewer queries than visual tokens ($n < m$) to verify whether equal cardinality is actually necessary for the reordering capability, or whether compression through queries (like Q-former's 32 queries) would also benefit from causal attention.

• Training data scaling: The paper notes that DeepSeek-OCR 2 uses "minimal differences" in training data (Section 4.1) but does not ablate the effect of the OCR 1.0 rebalancing (3:1:1 ratio) or the label refinement separately. These changes could contribute to performance differences independently of the architecture.

• Training stage contributions: The paper does not report performance after Stage 1 (encoder pretraining), Stage 2 (query enhancement), or Stage 3 (decoder specialization) separately, making it impossible to assess how much each training stage contributes to the final performance.

• Visual token budget scaling within DeepSeek-OCR 2: The paper explores token budget only through the multi-crop mechanism (0–6 crops, 256–1,120 tokens) but does not show performance at different budget points. This would reveal whether ordering efficiency gains saturate at lower budgets or continue to compound.

Critical Assessment

Does the evidence support the central claims?

Claim 1: DeepEncoder V2's causal reordering mechanism improves visual reading logic.

What the experiments demonstrate: The reading-order edit distance (R-order ED) improves from 0.085 to 0.057 (Table 1), and this improvement holds across all nine document types (Table 3). The production repetition rate (a proxy for ordering quality) decreases from 6.25% to 4.17% on user images and 3.69% to 2.88% on PDFs (Table 4).

What limits the demonstration: The paper provides no direct evidence that the ordering improvement comes specifically from the causal attention structure rather than from other architectural changes. The Qwen2-0.5B backbone (500M parameters, pretrained on text) has substantially more capacity and different pretraining than CLIP ViT (300M, pretrained on image-text pairs). The improved reading order could arise from:

• Better feature extraction due to a larger backbone (more parameters mean more expressive visual features, leading to better layout understanding even without causal reordering)
• Cross-modal knowledge transfer from the LLM backbone's text pretraining (the Qwen2 backbone may bring language-based reasoning about document structure that CLIP lacks)
• The learnable queries mechanism (even with bidirectional attention among queries, as in Q-former)
• The causal attention structure specifically (the paper's claimed mechanism)

Without a bidirectional query baseline, the paper cannot rule out any of these alternative explanations. The universal reading-order improvement across document types is suggestive but not conclusive, because a better feature extractor would also produce universal improvements—it would extract cleaner layout features from all document types, leading to better ordering regardless of the mechanism.

Moreover, the repetition rate improvement in Table 4 could result from the model being generally more accurate (producing correct text less likely to trigger repetition cascades) rather than specifically better at ordering. A model with 90% text accuracy will naturally have lower repetition than one with 85% accuracy, because accuracy failures often manifest as garbled output that the decoder repeats. The paper does not disentangle ordering quality from overall accuracy in the production setting.

Assessment: The evidence is consistent with the claim but does not isolate it. The reading-order improvements are real and substantial, but their causal origin in the causal attention mechanism specifically—rather than in increased encoder capacity or general architectural improvements—remains unproven by the reported experiments.

Claim 2: Causal reordering compensates for visual token compression, enabling high performance with low token budgets.

What the experiments demonstrate: DeepSeek-OCR 2 at 1,120 tokens outperforms multiple models using 5–7× more tokens (Table 1): InternVL3 (>7,000 tokens, 80.33%), Qwen2.5-VL-72B (>6,000 tokens, 87.02%), Gemini-2.5 Pro (1,120 tokens, 88.03%). At matched token budget (Gemini-2.5 Pro), DeepSeek-OCR 2 leads by 3.06 percentage points.

What limits the demonstration: These are cross-model comparisons where everything differs—architecture, training data, model scale, pretraining objectives, optimization recipes. The fact that model A with fewer tokens outperforms model B with more tokens does not mean that "ordering compensates for compression" within model A. It could equally mean that model A simply has a better architecture overall, better training data, or better alignment with the benchmark. The Gemini-2.5 Pro comparison at equal budget is the cleanest cross-model test and supports the claim, but is still confounded by all other differences between the models.

The paper never tests within its own architecture whether increasing token budget while keeping ordering quality constant (or vice versa) affects performance. A controlled ablation would be to train DeepSeek-OCR 2 variants with different numbers of local crops (varying the token budget from 256 to, say, 2,000+) and measure how accuracy scales, then compare this to DeepSeek-OCR's scaling curve. If DeepSeek-OCR 2's curve has a steeper slope (more accuracy per token), that would directly demonstrate ordering efficiency. Without this, the claim remains inferred from cross-model comparisons that are inherently confounded.

The claim also involves a subtle conceptual confusion. The paper argues that "better ordering allows the decoder to extract more information from fewer tokens." But information extraction is ultimately measured by downstream task accuracy, and accuracy depends on both ordering and content fidelity. The paper has not demonstrated that ordering can compensate for arbitrary compression losses—only that their specific model with their specific ordering mechanism achieves high accuracy at their specific token budget. The claim, as stated, suggests a general principle; the evidence supports a specific instance of that principle in one model family on one benchmark.

Assessment: The evidence demonstrates that DeepSeek-OCR 2 achieves strong performance at low token counts, but the causal link between "ordering quality" and "token efficiency" is not directly tested within a controlled experimental framework. The cross-model comparisons are suggestive but confounded.

Claim 3: The dual-stream attention mask enables the encoder to acquire causal reasoning capabilities.

What the experiments demonstrate: The model works. DeepSeek-OCR 2 achieves 91.09% overall accuracy (Table 1) and shows improved reading order across all document types (Table 3). The architecture trains successfully (Section 4.2) and produces semantically coherent reorderings as measured by reduced reading-order edit distance.

What limits the demonstration: The paper provides no evidence that the dual-stream mask specifically—as opposed to any mechanism that introduces sequential structure among queries—is necessary or beneficial. The mBART-style cross-attention alternative "failed to converge" (Section 3.2.2), which the paper reports as evidence favoring the prefix-based dual-stream design. But this tells us only that one alternative architecture failed, not that the dual-stream mask uniquely enables causal reasoning. Other alternatives (e.g., bidirectional queries with positional encodings, learned query ordering through reinforcement learning, attention-based reordering scores) are not tested.

The paper also provides no analysis of what the model actually learns. There are no visualizations of attention patterns to show that queries attend to semantically (rather than spatially) related visual tokens, no analysis of whether the learned ordering correlates with ground-truth reading order at the token level, and no probing experiments to determine whether the causal query representations encode layout structure information. The "causal reasoning capability" is inferred from output quality rather than directly demonstrated through analysis of internal representations.

The paper's report of the mBART failure is itself informative but under-specified. We are told that cross-attention "fails to converge," but not: at what loss value? After how many training steps? On what training data? With what hyperparameter configurations? "Fail to converge" could mean the loss oscillated, or diverged, or converged to a poor local optimum. Without details, the negative result cannot inform future architectural design.

Assessment: The dual-stream mask is a novel architectural mechanism that demonstrably produces a working model, but the paper does not demonstrate that this specific mask structure is responsible for the model's causal reasoning capabilities. The claim that the mask "enables" causal reasoning is consistent with the results but not experimentally isolated.

Claim 4: The cascade of two 1D causal reasoners represents progress toward genuine 2D reasoning.

What the experiments demonstrate: The model performs well on document OCR, which requires understanding 2D layout structures (tables, multi-column text, formula spatial arrangements). The improvements in formula CDM (+6.17, Table 1) and table TEDs (+2.50) suggest better handling of 2D-structured content.

What limits the demonstration: The paper acknowledges this is a preliminary validation on one task type (document reading) and that "achieving this goal remains a long journey" (Section 6.1). The experiments do not test whether the two-stage causal cascade generalizes to other forms of 2D reasoning—visual question answering about spatial relationships, diagram understanding, chart interpretation, or scene comprehension. Document reading is a specific case where the mapping from 2D to 1D (reading order) is well-defined; many 2D reasoning tasks do not have a natural 1D linearization. Whether the architecture would transfer to open-ended visual reasoning tasks where "what order should I process this in?" lacks a clear answer is completely untested.

Moreover, the paper does not demonstrate that the two stages actually perform distinct, complementary forms of reasoning as hypothesized. The encoder could be performing generic feature extraction (like a standard ViT) while the decoder handles both layout understanding and content extraction, with the reading-order improvement coming entirely from better feature quality rather than from a division of reasoning labor. Without probing or analysis of intermediate representations, the two-stage decomposition remains a design intention rather than an empirically verified property of the trained model.

Assessment: The claim is explicitly presented as a research hypothesis with preliminary validation, not as an established finding. The experiments provide existence proof that a two-stage causal pipeline can work for document reading, but they do not demonstrate that the specific form of the two-stage decomposition (reading logic reasoning + task reasoning) is actually occurring, nor that it generalizes beyond the document domain. The paper's cautious framing ("progress toward," "remains a long journey") is appropriate.

Genuine weaknesses in the experimental design

Missing critical ablation: bidirectional queries. The single most important experiment not conducted is a DeepEncoder V2 variant where the causal flow queries use bidirectional self-attention (like DETR and Q-former) instead of the causal triangular mask, keeping all other aspects identical (backbone, equal cardinality, visual prefix). This would directly test whether causal attention specifically contributes to reordering capability or whether the improvements come from introducing learnable queries and a larger backbone. Without this ablation, the paper's central claim—that causal reordering is the mechanism of improvement—remains unverified.

No within-model token budget scaling analysis. The paper emphasizes the 1,120 token budget as a competitive advantage but never shows how performance varies with budget within DeepSeek-OCR 2. A scaling curve (accuracy vs. number of local crops, from 0 to 6) would reveal the marginal benefit of additional tokens and allow comparison with DeepSeek-OCR's scaling behavior. This is a straightforward experiment using the same multi-crop mechanism the model already employs; its absence is notable.

Confounded comparison against the primary baseline. The comparison against DeepSeek-OCR (Table 1) is the cleanest in the paper, but still confounded: DeepEncoder V2 replaces the CLIP component with a model that has ~67% more parameters (500M vs. 300M), different pretraining (LLM vs. vision-language), and a fundamentally different architecture (decoder-only vs. ViT). The paper does not control for parameter count by, for example, testing a larger CLIP ViT with comparable parameters to Qwen2-0.5B, or testing a smaller LLM backbone closer to CLIP's scale.

Single benchmark evaluation. All formal evaluations use OmniDocBench v1.5. While this benchmark is diverse (9 document types, bilingual), it represents a single evaluation methodology and a single task domain (document parsing). The paper demonstrates no generalization to other OCR benchmarks (e.g., FUNSD for form understanding, SROIE for receipt OCR, DocVQA for document question answering), general visual reasoning benchmarks (MMBench, MME, SEED-Bench), or even simple sanity checks (accuracy on clean, single-column text documents vs. complex layouts to verify the ordering mechanism only engages when needed).

No statistical significance testing on a 1,355-sample test set. The OmniDocBench test set contains 1,355 pages, but broken down by document type, some categories are quite small (the paper doesn't provide per-category sample counts, but with 9 categories averaging ~150 samples each and some likely much smaller). The 3.73% overall improvement is on the full set and is substantial, but per-category improvements (e.g., Academic Paper R-order: 0.021 → 0.013, an improvement of 0.008) may not be statistically significant at small sample sizes. No confidence intervals or significance tests are reported.

Production metrics are narrow and unvalidated. The repetition rate metric in Table 4 captures only one failure mode. A model could improve repetition rate by becoming more conservative (generating less text overall, avoiding complex layouts) without actually being more accurate. The paper does not validate repetition rate against ground truth (e.g., by showing that lower repetition correlates with higher accuracy on a labeled subset) or report complementary production metrics (throughput, latency, output length distribution, hallucination rate).

No comparison against DeepSeek-OCR with the same decoder-only training. DeepSeek-OCR 2 benefits from Stage 3 training (decoder specialization with frozen encoder, Section 4.2.3), which "more than doubles the training speed" and allows processing additional data. It's unclear whether DeepSeek-OCR was trained with an equivalent Stage 3 or whether DeepSeek-OCR 2's advantage partly comes from additional training iterations and data consumption enabled by the frozen-encoder stage, rather than from the encoder architecture.

Experiments that would have strengthened the paper

1. Bidirectional query baseline: The most impactful missing experiment. Train DeepEncoder V2 with bidirectional (all-to-all) self-attention among queries instead of causal, keeping everything else identical. This isolates the causal mechanism.

2. Parameter-matched baseline: Replace Qwen2-0.5B with a ViT of comparable parameter count (e.g., ViT-Large at ~300M with additional layers to reach 500M, or scale up CLIP ViT), keeping the standard raster-scan ordering. This isolates backbone capacity from the reordering mechanism.

3. Token budget scaling curves: For both DeepSeek-OCR and DeepSeek-OCR 2, plot accuracy vs. number of visual tokens (varying crop count). This would show: (a) whether DeepSeek-OCR 2's curve is steeper (better ordering efficiency), (b) where each model plateaus, and (c) the minimum token budget where DeepSeek-OCR 2 matches DeepSeek-OCR at full budget.

4. Attention visualization and probing: Analyze where causal flow queries attend—do they follow reading order at the attention level? Probe query representations for layout structure information (can a linear classifier predict reading order position from query embeddings?).

5. Multi-benchmark evaluation: At minimum, a second OCR benchmark (FUNSD, SROIE) and one general vision benchmark (e.g., MMBench) to test generalization beyond document parsing.

6. Training stage ablations: Report performance after Stage 1, Stage 2, and Stage 3 to quantify the contribution of each training phase to the final accuracy. This would also reveal whether the causal reordering capability emerges primarily during encoder pretraining (Stage 1) or during joint encoder-decoder training (Stage 2).

7. Query cardinality ablation: Test with fewer queries than visual tokens ($\frac{n}{2}$ or $\frac{n}{4}$) to determine whether equal cardinality is necessary or whether compression through causal queries is actually more efficient.

8. Cross-decoder generalization test: Take a trained DeepEncoder V2 and pair it with a different decoder (e.g., a generic LLaMA rather than the DeepSeek-MoE decoder) to test whether the encoder's reordering is architecture-specific or generalizes across decoders. This would support the claimed separation of concerns between encoder ordering and decoder reasoning.

Conditional nature of the claims

The paper's claims hold under conditions that the experiments cannot fully verify:

• The causal reordering claim holds if and only if the reading-order improvements originate specifically from causal attention rather than from increased backbone capacity or the introduction of learnable queries. This condition is plausible but unverified.

• The token efficiency claim holds if and only if DeepSeek-OCR 2's performance advantage over higher-budget models persists after controlling for other factors (pretraining data quality, model scale, training recipes). The cross-model comparisons cannot establish this control.

• The two-stage 2D reasoning claim holds if and only if the architecture generalizes beyond document OCR to visual reasoning tasks where the 1D linearization is less natural. The current experiments offer no evidence for or against this generalization.

• The practical readiness claim holds if and only if the reduced repetition rate genuinely reflects improved accuracy rather than changed output behavior (e.g., shorter responses, avoidance of challenging layouts). This condition is not validated.

The paper's strongest contribution—documented and verifiable—is that replacing a CLIP-based encoder with an LLM-style encoder incorporating learnable causal queries produces a substantial accuracy improvement on document OCR (3.73% on OmniDocBench v1.5). The paper's more ambitious claims about causal reordering as a mechanism, token efficiency as a principle, and two-stage 1D reasoning as a path to 2D understanding are hypotheses supported by the aggregate results but not experimentally isolated or validated beyond the document reading domain.

6. Limitations and Trade-offs

6.1 Critical Missing Ablation: Causal Attention versus Bidirectional Queries

The assumption or constraint. The paper's central claim is that causal attention among learnable queries—the triangular LowerTri(n) mask in Equation 1—is the mechanism that enables semantic reordering of visual tokens, and that this reordering is responsible for the 3.73% improvement over DeepSeek-OCR. However, the paper never compares DeepEncoder V2 against a variant where the query tokens use bidirectional self-attention (all-to-all, like DETR's object queries and Q-former's compression queries) instead of causal attention. The architecture simultaneously changes three things relative to DeepSeek-OCR: (1) the backbone from CLIP ViT (300M) to Qwen2-0.5B (500M), (2) the introduction of learnable queries with equal cardinality, and (3) the causal attention pattern among those queries. The paper provides no experiment that isolates which of these changes drives the performance improvement. The only alternative architecture tested—an mBART-style encoder-decoder with cross-attention—"fail[s] to converge" (Section 3.2.2), but this tests a fundamentally different architectural family, not a controlled variation of the causal mask.

The consequence. Without a bidirectional-query baseline, none of the paper's claims about the specific role of causal attention are empirically grounded. The 3.73% overall improvement and the consistent reading-order gains across all nine document types (Table 3) could equally be explained by: (a) the larger backbone providing better visual features, (b) the introduction of any learnable queries (regardless of attention pattern) enabling more flexible visual representation, or (c) the Qwen2-0.5B's text pretraining providing cross-modal transfer for document structure understanding. The causal attention mechanism might be entirely incidental to the results—the model could be learning equivalent reordering through bidirectional query interactions combined with positional encodings, with the causal mask serving only as a training regularization rather than as a functionally necessary constraint for reordering. This matters for both scientific understanding and practical deployment: if the gains come primarily from the larger backbone, future work should invest in scaling encoder capacity; if they come from the queries, bidirectional queries might work equally well with simpler training (no causal dependency to maintain); if they come specifically from causal structure, then the architectural innovation is genuinely validated. The paper cannot distinguish these possibilities.

What evidence exists in the paper. Section 3.2.2 reports the mBART failure as evidence that the prefix-based design is necessary ("We hypothesize this failure stems from insufficient visual token interaction when isolated in a separate encoder"), but this is evidence for the prefix concatenation strategy, not for causal attention specifically. The reading-order improvement (Table 1: R-order ED 0.085 → 0.057) demonstrates that ordering improved, but cannot distinguish whether the ordering mechanism is causal or bidirectional, since a bidirectional query set could also learn to produce an ordered output by encoding ordering information in the query representations themselves (e.g., through learned position-specific embeddings). No attention pattern analysis, probing experiments, or controlled ablation of the LowerTri(n) mask is reported anywhere in the paper.

Mitigation status. Not addressed. The paper does not acknowledge this as a missing experiment or discuss the confound between backbone capacity, query introduction, and causal attention. Future work that the paper does not propose but that would be essential before accepting the causal flow hypothesis: training identical DeepEncoder V2 architectures with (a) LowerTri(n) mask (causal), (b) 1_{n×n} mask (bidirectional among queries), and (c) no queries (pure bidirectional visual encoding with LLM backbone) and comparing their reading-order performance.

---

6.2 Single Benchmark, Single Task Domain: No Evidence of Generalization Beyond Document OCR

The assumption or constraint. All formal evaluations use exactly one benchmark: OmniDocBench v1.5 (Section 5.1), comprising 1,355 document pages across 9 document types in Chinese and English. The paper explicitly adopts "document reading as our primary experimental testbed" (Section 1) and justifies this choice by noting that documents "present rich challenges including complex layout orders, intricate formulas, and tables" that "inherently carry causal visual logic." However, the paper's broader claims extend well beyond document OCR: Section 6.1 frames the two-stage cascade as progress toward "genuine 2D reasoning," and Section 6.2 envisions the LLM-style encoder as a step toward "native multimodality" and "unified omni-modal encoding" spanning "images, audio, text" and "more diverse scenarios." These claims imply that the architecture's effectiveness should generalize, but the paper provides zero evidence on any non-document visual task.

The consequence. Document reading is a uniquely favorable domain for the paper's architectural hypothesis for one specific reason: the mapping from 2D layout to 1D sequence has a well-defined ground truth (natural reading order). The model can be trained and evaluated against an objective correct ordering, and failure to follow reading order produces measurable edit distance penalties. Most visual reasoning tasks lack this property. Consider: what is the "correct" order in which to process the regions of a photograph for visual question answering? For scene understanding? For diagram comprehension? For chart interpretation? There is no well-defined 1D linearization—the optimal processing order may depend on the question being asked, the task being performed, or may not even exist (some visual reasoning is inherently non-sequential). The paper's central contribution—causal reordering into a semantic sequence—may be solving a problem (2D-to-1D mapping) that only exists for document-like visual tasks. On general visual reasoning, the raster-scan ordering the paper criticizes as "unwarranted inductive bias" might be no worse than any learned alternative, because the concept of a "correct" ordering may be ill-defined. The paper provides no test of whether the causal flow mechanism helps, hurts, or is neutral for general visual tasks where ordering is arbitrary.

What evidence exists in the paper. The evaluation is entirely confined to OmniDocBench v1.5 (Table 1, Table 2, Table 3) and production OCR metrics (Table 4). No results are reported on any general visual reasoning benchmark (MMBench, MME, SEED-Bench, MMMU, DocVQA for document question answering specifically, ChartQA for chart reasoning, or even simpler tasks like image captioning on MS-COCO). The paper does not even evaluate on other OCR benchmarks (FUNSD for form understanding, SROIE for receipt parsing, PubTables-1M for table structure recognition) that would test whether the reading-order improvements transfer across OCR sub-domains.

Mitigation status. The authors partially acknowledge this in Section 6.1: "achieving this goal remains a long journey" and "we will continue to refine this architecture and explore its effectiveness on general visual reasoning tasks in future work." Section 7 similarly notes that "optical text reading... constitutes only a small part of the broader visual understanding landscape." These are acknowledgments of scope limitation, not mitigation. The paper presents no evidence on non-document tasks and makes no architectural arguments for why the causal reordering mechanism would transfer. A practitioner evaluating whether to adopt this architecture for a non-OCR vision task has no data to inform that decision.

---

6.3 No Within-Model Token Budget Scaling Analysis: The Efficiency Claim Rests Entirely on Cross-Model Comparisons

The assumption or constraint. A central claim of the paper is that semantic reordering enables high accuracy at low visual token counts—that "better ordering allows the decoder to extract more information from fewer tokens" (insight discussed in Section 4), that DeepSeek-OCR 2 "maintains high compression rates of visual tokens while ensuring superior performance" (Section 5.1), and that the model achieves these results "while using the smallest upper limit of visual tokens" among compared end-to-end models (Table 1 commentary). The evidence for this claim comes entirely from cross-model comparisons: DeepSeek-OCR 2 at 1,120 tokens outperforms InternVL3 at 7,000+ tokens, Qwen2.5-VL-72B at 6,000+ tokens, and Gemini-2.5 Pro at a matched 1,120 tokens. The paper never tests within its own architecture how performance scales with visual token budget.

The consequence. Cross-model comparisons confound token budget with every other difference between models—pretraining data scale and composition, model architecture, total parameter count, optimization hyperparameters, training duration, and language model decoder capacity. The fact that model A with fewer tokens outperforms model B with more tokens does not demonstrate that ordering quality substitutes for token quantity. It could equally mean that model A has a better decoder, was trained on better data, has more effective pretraining, or benefits from any of a dozen other confounds. The Gemini-2.5 Pro comparison at matched budget (1,120 tokens) is the cleanest cross-model test—DeepSeek-OCR 2 leads by 3.06 percentage points (91.09% vs. 88.03%, Table 1)—but even this doesn't isolate ordering from other factors. The paper's own baseline (DeepSeek-OCR vs. DeepSeek-OCR 2 at roughly matched budget, 1,156 vs. 1,120 tokens) supports the claim that the new architecture is better, but doesn't show that this advantage comes specifically from improved token efficiency rather than from generally better encoding.

What a practitioner actually needs to know: If I have a fixed compute budget for inference, should I invest in more visual tokens with a standard encoder, or fewer tokens with this causal reordering encoder? The paper provides no data to answer this. A within-model scaling curve—accuracy of DeepSeek-OCR 2 at 256, 400, 600, 800, 1,000, and 1,120 tokens—would reveal the marginal value of additional tokens and enable direct comparison against DeepSeek-OCR's scaling curve. If DeepSeek-OCR 2's curve has a steeper slope (more accuracy gained per additional token), the ordering efficiency claim is validated. If it has a similar or shallower slope, the gains come from a higher intercept (better base encoding), not from steeper scaling (better token utilization). This experiment is straightforward—the multi-crop mechanism already generates variable token counts from 256 to 1,120 depending on the number of local crops—yet the paper does not report it.

What evidence exists in the paper. The comparison of V-token max across models appears in Table 1 (where DeepSeek-OCR 2's 1,120 is the lowest among end-to-end models except DeepSeek-OCR's 1,156) and Table 2 (where DeepSeek-OCR 2's 0.100 overall ED at 1,120 tokens beats Gemini-3 Pro's 0.115 at matched budget and Seed-1.8's 0.106 at 5,120 tokens). These are cross-model comparisons. The paper acknowledges that the lower token budget might cause problems for specific document types (Section 5.2: "the lower upper limit of visual tokens may affect the recognition of text-super-rich newspapers") but does not test this hypothesis within its own model family by, for instance, evaluating newspaper accuracy with and without additional local crops. No token budget scaling curve is reported for either DeepSeek-OCR or DeepSeek-OCR 2.

Mitigation status. Not addressed. The paper treats the 1,120 token budget as a fixed design target (matching Gemini-3 Pro's budget, Section 3.2.3) and reports only endpoint performance at that budget. The multi-crop mechanism that generates variable token counts is used for training but is not exploited for analysis. The paper's suggestion that newspaper performance could be improved by "increasing the number of local crops" (Section 5.2) implicitly acknowledges that token budget matters, but this is presented as a future fix rather than investigated as a scaling relationship.

---

6.4 Production Metrics Capture Only One Failure Mode and Are Not Validated Against Accuracy

The assumption or constraint. Section 5.3 evaluates "practical readiness" using exactly one metric: repetition rate, defined as "the percentage of outputs containing repetitive or semantically garbled text." The paper states that "since ground truth is unavailable in production environments, we focus primarily on repetition rate as our key metric" (Section 5.3). The reduction in repetition rate (6.25% → 4.17% on online images, 3.69% → 2.88% on PDFs, Table 4) is presented as evidence that "DeepSeek-OCR 2 demonstrates markedly improved practical readiness" and that these results "validate the effectiveness of the DeepSeek-OCR 2 architecture, particularly its logical visual comprehension capabilities."

The consequence. Repetition rate is a narrow signal that captures only one specific failure mode: when the decoder gets stuck in a repetitive loop (often triggered by garbled or misordered visual tokens that confuse the language model). It does not capture: (a) incorrect but non-repetitive text (the model confidently outputs wrong content without looping), (b) missed content (the model skips sections of the document entirely), (c) hallucinated content (the model inserts text not present in the image), (d) misformatted output (correct text but wrong structure), or (e) degraded output quality in ways that don't manifest as repetition. A model could achieve a low repetition rate by being more conservative—generating shorter output overall, avoiding complex layouts that trigger confusion, or defaulting to simple responses when uncertain—without actually being more accurate. The reduction from 6.25% to 4.17% could reflect the model learning to bail out of confusing situations (producing an "unable to parse" response rather than looping) rather than learning to correctly process those confusing situations. This is a genuine concern for an OCR system serving LLM pipelines: if the OCR under-generates (misses content) rather than over-generates (repeats), the downstream LLM receives incomplete information, potentially causing worse outcomes than receiving repetitive but complete information that the LLM might partially recover from.

More broadly, evaluating production quality without ground truth is inherently challenging, but a single unvalidated metric is insufficient for the paper's claim of "markedly improved practical readiness." Production-readiness claims typically require multi-dimensional evaluation: accuracy proxies (does the OCR output enable correct downstream task completion?), latency (is the model fast enough for real-time serving?), throughput (can it handle the production request volume?), robustness to input variation (image quality, resolution, language mix), and failure mode characterization (what kinds of documents does it fail on, and how gracefully?).

What evidence exists in the paper. Table 4 reports repetition rates on two production datasets. No other production metrics are provided. The paper does not validate repetition rate against ground truth on any labeled subset (e.g., showing that on a small set of manually annotated production samples, lower repetition correlates with higher accuracy). There is no comparison of output length distributions, vocabulary diversity, or downstream task accuracy between DeepSeek-OCR and DeepSeek-OCR 2. No latency or throughput numbers are reported for either production setting. The paper provides no characterization of what kinds of production documents benefit from the reordering mechanism and which do not.

Mitigation status. Not addressed. The paper interprets the repetition rate reduction as validating "logical visual comprehension capabilities" (Section 5.3) without discussing the metric's limitations or providing complementary evidence. The production evaluation is presented as a practical validation, but the narrowness of the metric and the absence of ground-truth validation means it functions more as a sanity check (the model doesn't catastrophically loop) than as a genuine demonstration of production readiness.

---

6.5 No Analysis of Internal Representations: The Claimed Causal Reordering Mechanism Is Unobserved

The assumption or constraint. The paper's central mechanistic claim is that the causal flow queries learn to semantically reorder visual information before it reaches the decoder—that the encoder "imposes semantic ordering and distilling on visual features" (Section 3.2.2), that the queries "progressively build a semantically ordered sequence" (elaborated in Section 3.4), and that this reordering is what enables the decoder to more effectively process visual content. However, the paper provides no direct evidence that the causal flow queries actually learn to perform semantic reordering. All evidence for the reordering capability is behavioral: the output has lower reading-order edit distance (Table 1, Table 3), and the production system has lower repetition rate (Table 4). The internal mechanism—what the queries attend to, what information they encode, whether their ordering corresponds to semantic reading order—is entirely unexamined.

The consequence. Without analysis of internal representations, the paper cannot distinguish between two fundamentally different models of how the architecture achieves its improvements:

Model A (the paper's claimed mechanism): The causal flow queries genuinely reorder visual information. Early queries attend to and encode content that should come first in reading order (titles, headers). Later queries build on this to encode subsequent content (body text, captions). The sequence q_0, q_1, ..., q_{n-1} is a causally-ordered representation where the ordering carries semantic meaning, and the decoder benefits specifically from this ordering.

Model B (an alternative explanation): The queries function as a feature refinement mechanism. The causal attention provides a form of iterative processing where each query refines the visual representation without necessarily reordering it. The decoder receives a sequence that is better than the raw visual tokens not because it's semantically reordered, but because it's been processed through a deeper, more sophisticated feature extraction pipeline (500M parameters of transformer layers with full visual context access). The reading-order improvement comes from better content understanding (cleaner text features, better layout feature extraction) enabling the decoder to more accurately determine reading order from content, rather than from the encoder explicitly reordering content.

Model A and Model B have different implications: if Model A is correct, the architecture is genuinely performing a novel form of visual reasoning, and further improvements should focus on enhancing the reordering capability (more queries, multi-pass reordering, explicit ordering supervision). If Model B is correct, the architecture is an effective but conventional feature extractor, and further improvements should focus on scaling the encoder's capacity for better feature quality. The paper's experiments cannot distinguish these models.

Concretely, several analyses would directly test the reordering claim: (a) attention maps showing which visual tokens each causal flow query attends to (do early queries systematically attend to title-region tokens? do late queries attend to body-text tokens?), (b) correlation between query position and ground-truth reading order position (can a linear probe predict reading order position from query embeddings?), (c) comparison of query-to-visual attention patterns against a bidirectional query baseline (are the attention patterns more "ordered" with causal constraints?), or (d) intervention experiments (what happens to output if you shuffle the query order before feeding to the decoder?—if reordering is real, shuffled queries should degrade performance; if queries are just refined features, order should matter less). None of these analyses are performed.

What evidence exists in the paper. The behavioral evidence is consistent with reordering but does not require it. Lower R-order ED (Table 1, Table 3) means the output reading order is better; it does not mean the encoder caused this improvement by reordering visual tokens. The improvement in formula CDM (+6.17, Table 1) is consistent with better linearization of 2D structures, but could also reflect better feature extraction enabling the decoder to handle formula layout internally. The mBART-style cross-attention failure (Section 3.2.2) tells us the prefix design is necessary for convergence, not that the converged model performs reordering.

Mitigation status. Not addressed. The paper makes no attempt to analyze internal representations, attention patterns, or query behavior. The term "causal flow" is used as a design intention and an output descriptor (the output has better reading order), but the intermediate step—do the queries actually create causal flow at the representational level?—is never examined. This is a significant gap for a paper whose primary contribution is a novel architectural mechanism: the mechanism is specified in the architecture, but whether it actually operates as intended in the trained model is never verified.

---

6.6 Model Does Not Improve on the Hardest Cases: Newspaper Text Recognition Degrades and Magazine Reading Order Barely Improves

The assumption or constraint. The paper claims that causal reordering improves visual understanding across document types, particularly for "images with complex layouts" that "exhibit causally-informed sequential processing" (Section 1). However, the detailed results in Table 3 reveal that the architecture's benefits are uneven and that there exist document types where DeepEncoder V2 either degrades performance or provides negligible improvement—specifically the categories that should benefit most from reordering (newspapers and magazines, which have the most complex, non-linear layouts).

The consequence. On Newspaper documents, DeepSeek-OCR 2's text recognition edit distance degrades from 0.131 to 0.139 (+0.008, Table 3)—the new architecture is worse at extracting text from newspapers than the baseline. The paper attributes this to "the lower upper limit of visual tokens" and "insufficient training data—our training data contains only 250k relevant samples" (Section 5.2). But these explanations undermine the paper's central narrative: if the causal reordering mechanism genuinely creates more efficient token usage, the lower token budget should be compensated by better ordering. The fact that it isn't—that newspapers specifically show degradation—suggests that the reordering mechanism may have a complexity ceiling beyond which it cannot effectively linearize layout, and that this ceiling is hit by newspaper layouts before the token budget becomes the binding constraint. On Magazine documents, reading-order edit distance shows the smallest improvement across all nine categories (0.101 → 0.100, a reduction of only 0.001, Table 3). The R-order ED remains at 0.100—an order of magnitude worse than simpler documents (Academic Paper: 0.013, Research Report: 0.011). So for the document types with the most complex layouts—precisely where the gap between raster-scan ordering and semantic reading order is widest, and where the causal reordering mechanism should provide the greatest benefit—the architecture either degrades (newspaper text) or barely improves (magazine reading order).

This has significant practical implications: an organization deploying DeepSeek-OCR 2 for document processing cannot assume uniform improvement. Documents with simple linear layouts (academic papers, research reports) will see strong gains; documents with dense, multi-element layouts (newspapers, magazines) may see minimal gains or even regressions. The architecture's claimed strength—handling "complex layout orders" (Section 1)—is exactly where it shows its weakest empirical results.

What evidence exists in the paper. Table 3 provides the per-category breakdown. The paper explicitly acknowledges the newspaper issue (Section 5.2): "there are also notable weaknesses, such as newspapers, where it performs >0.13 ED" and attributes it to token budget and data scarcity. The magazine reading-order near-stagnation (0.101 → 0.100) is visible in the data but not commented on in the text. The paper does not discuss whether magazine and newspaper layouts share structural properties that the causal reordering mechanism fundamentally struggles with (e.g., very high text density per unit area, multiple independent reading paths, non-hierarchical layout structures where "reading order" is inherently ambiguous).

Mitigation status. Partial. The paper proposes two mitigations (Section 5.2): "increasing the number of local crops" (to raise token budget) and presumably adding more training data for underrepresented categories. However, increasing local crops increases token budget, which partially contradicts the paper's efficiency narrative (the claim that fewer tokens can work better). If newspaper performance requires more tokens, then the ordering mechanism's compensation for token compression has limits, and those limits are hit by document types that should theoretically benefit most. The paper's mitigation proposals are engineering solutions (more data, more compute) rather than architectural improvements to the reordering mechanism, suggesting the fundamental challenge of linearizing highly complex 2D layouts is not addressed by the current causal flow design.

7. Implications and Future Directions

How This Work Changes the Landscape

A new design dimension for vision encoders: learned semantic reordering as a first-class architectural primitive. Before DeepEncoder V2, the dominant assumption in vision-language model design was that visual tokens reach the decoder in raster-scan order, and the decoder's job is to reconstruct semantic relationships from this spatially-imposed sequence. Every major VLM architecture—LLaVA, Qwen-VL, InternVL, DeepSeek-OCR itself—inherits this ordering without questioning it. The encoder's role was feature extraction; ordering was not a design dimension at all.

DeepEncoder V2 changes this by demonstrating that interposing a causal reordering stage between feature extraction and linguistic reasoning produces measurable, consistent improvements in tasks where spatial order and semantic order diverge. The 3.73% absolute improvement on OmniDocBench v1.5 is substantial for an architectural change that touches only the encoder component while preserving the training data engine, decoder, and overall pipeline. The universal reading-order improvement across all nine document types (Table 3)—including gains of 0.059 ED on colorful textbooks and 0.054 on notes, precisely the categories where raster order most poorly approximates reading order—establishes that the reordering capability is learned and generalizable, not an artifact of specific document types.

This is not a paradigm shift on the scale of "attention replaces recurrence" or "pretraining replaces task-specific architectures." It is narrower and more targeted: a reframing of what a vision encoder's responsibilities are in multimodal architectures. The encoder is no longer just an information extractor; it is now an information extractor that also decides the sequence in which extracted information should be consumed. This reframing opens a design dimension that was previously invisible to the field—not because it was impossible, but because the default raster-scan ordering was so deeply embedded in VLM architectures that questioning it was not on the research agenda.

Reconciling the CLIP-versus-LLM-as-encoder debate. The paper provides the strongest empirical evidence to date that an LLM-style architecture can function as an effective vision encoder—not just as a modality-agnostic processor that happens to accept visual tokens (as in Fuyu or Chameleon), but as a component that contributes distinct capabilities (causal reordering) that a standard ViT-based encoder cannot provide. Prior work on LLM-based multimodal initialization (Section 2.3) demonstrated that LLM weights provide useful starting points for visual tasks, but did not show that the LLM architecture itself enables novel visual processing capabilities. DeepEncoder V2 demonstrates that the causal attention inherent to decoder-only LLMs—typically considered a limitation for vision (which benefits from bidirectional processing)—can be an asset when applied to a parallel stream of learnable queries alongside bidirectional visual processing. This partially resolves the apparent tension between "LLMs are good for text, ViTs are good for images" by showing that a hybrid approach (bidirectional for visual tokens, causal for reasoning tokens within the same architecture) captures the strengths of both.

Shifting optimization effort from decoder scaling to encoder sophistication. A subtle but important implication: the paper demonstrates that investing in encoder intelligence (better ordering) can substitute for decoder resources (fewer visual tokens, less decoder capacity wasted on disentangling spatial artifacts). The comparison against models using 5–7× more visual tokens (Table 1) suggests that the encoder-decoder boundary is not a fixed design point but a locus of tradeoff. Research programs that focus primarily on scaling decoder capacity or increasing visual token counts may be leaving gains on the table that could be captured through better encoder-side processing. This does not make decoder scaling obsolete—the paper's own decoder is unchanged from DeepSeek-OCR—but it adds encoder design as a complementary scaling axis that the field has underexplored.

What becomes more attractive as a research direction. Improving visual token ordering—through better causal mechanisms, multi-pass reordering, ordering-specific training objectives, or explicit reading-order supervision—becomes a newly visible research subfield. The paper's observation that equal cardinality ($n = m$) may be insufficient for "multiple re-examinations and multi-hop reordering" (Section 6.1) directly points toward architectures with $n > m$ queries, which would represent a genuine expansion beyond the current design. The dual-stream attention mask design pattern (bidirectional for content, causal for reasoning, asymmetric cross-attention) becomes a template that other researchers can instantiate with different backbones, different modalities, and different task-specific query designs.

What becomes less urgent. Further optimization of raster-scan-based architectures for document understanding. If semantic reordering provides a 3.73% gain and universal reading-order improvement while preserving token efficiency, the returns to incremental improvements in spatial-order processing (better positional encodings for raster-scan, larger ViT backbones for finer feature extraction on documents) are relatively diminished. The ceiling on raster-scan approaches is fundamentally bounded by the spatial-semantic mismatch; no amount of better feature extraction from a raster-ordered sequence can fully compensate for the ordering being wrong to begin with. Research effort is better directed at reordering mechanisms than at making raster order work slightly better.

Follow-Up Research This Work Enables

Causal versus bidirectional query ablation: isolating the mechanism. The single most important missing experiment in this paper is a controlled comparison between DeepEncoder V2 with the causal triangular mask ($\text{LowerTri}(n)$) and an identical architecture where the queries use bidirectional self-attention ($\mathbf{1}_{n \times n}$). Training both variants on identical data with an identical training pipeline and comparing their reading-order performance would definitively answer whether causal attention specifically—or the introduction of learnable queries generally—drives the reordering improvement. A strong follow-up would measure not just output metrics (reading-order ED, overall accuracy) but also internal behavior: attention pattern entropy (are causal queries' attention patterns more structured and position-dependent than bidirectional queries'?), position-accuracy correlation (can a linear probe predict a query's position in the sequence from its embedding, and is this correlation stronger for causal queries?), and robustness to query shuffling at inference time (does shuffling the causal query order degrade output more than shuffling bidirectional query order?). A negative result—bidirectional queries matching or exceeding causal queries—would reframe DeepEncoder V2's contribution as demonstrating the value of learnable query-based reordering generally, not causal reordering specifically.

Multi-pass reordering with n > m queries: testing the re-fixation hypothesis. The paper explicitly identifies a limitation: "to enable multiple re-examinations and multi-hop reordering of visual content, we may need substantially longer causal flow tokens than the original visual token sequence" (Section 6.1). This immediately suggests an architecture with $n = k \times m$ queries for $k > 1$—allowing each visual token to be "revisited" at multiple positions in the causal sequence. A concrete experiment: train DeepEncoder V2 variants with 1×, 2×, and 4× queries relative to visual tokens ($k \in \{1, 2, 4\}$). The prediction would be that larger $k$ enables more accurate reading order on complex layouts (magazines, newspapers—the categories where current DeepEncoder V2 shows minimal improvement or degradation) because the model can represent the same visual content at multiple semantic positions (e.g., a figure appearing both when first encountered and when its caption is processed). The cost is increased encoder computation and more visual tokens fed to the decoder (since all $k \times m$ queries reach the decoder), trading off the paper's token efficiency claims against reordering quality. This experiment would directly test whether the equal-cardinality design is a fundamental requirement or a pragmatic starting point, and whether the "cascade of two 1D reasoners" framework genuinely supports multi-hop visual reasoning or saturates at single-pass reordering.

Cross-domain stress testing: document ordering on natural images and general VQA. The paper's evaluation is confined to document OCR, where a ground-truth reading order exists. To determine whether the causal reordering mechanism has value beyond document-specific tasks, a strong follow-up would evaluate DeepEncoder V2 on benchmarks where ordering is less well-defined: (a) Visual Question Answering (e.g., VQAv2, GQA) to test whether learned ordering helps when no single correct order exists; (b) Chart and diagram understanding (e.g., ChartQA, DVQA) where 2D spatial structure matters but "reading order" is task-dependent; (c) Natural image captioning (e.g., MS-COCO, NoCaps) where raster-scan order is arguably reasonable and reordering might provide minimal benefit (a near-zero improvement would actually be an informative result, establishing a boundary condition for when causal reordering matters); and (d) Document VQA (e.g., DocVQA, InfographicVQA) which sits between document parsing and visual reasoning, requiring both reading order and task-specific visual attention. The key measurement would be whether the reading-order improvement on OmniDocBench translates to downstream task accuracy improvements or whether good reading order on document parsing is a specialized capability that does not generalize to tasks requiring flexible, question-conditioned visual attention.

Explicit reading-order supervision versus implicit learning. DeepEncoder V2 learns to reorder visual tokens entirely through the language modeling objective—there is no explicit loss term encouraging queries to follow ground-truth reading order. The model discovers reordering as a useful intermediate representation for the downstream task. An open question: would explicit supervision on reading order (e.g., an auxiliary loss that encourages the causal flow query sequence to match ground-truth reading order at the token level) improve performance, particularly on the document types where DeepEncoder V2 currently struggles (newspapers, magazines)? A follow-up could add a lightweight reading-order prediction head to the causal flow queries (predicting the ground-truth reading order position for each query) and compare performance against the purely implicit approach. The hypothesis would be that explicit supervision helps on complex layouts where the language modeling signal is too weak to guide the encoder toward correct ordering, and hurts on simple layouts where implicit learning already finds the optimal order and the auxiliary loss adds unnecessary constraints. This would inform whether reordering is best treated as a latent capability that emerges from task objectives or as an explicit capability that should be directly trained.

Encoder-decoder co-adaptation: does the reordering transfer across decoders? DeepEncoder V2 is trained jointly with the DeepSeek-MoE decoder (Stage 2) and then the decoder is specialized with the encoder frozen (Stage 3). This raises the question: is the learned reordering specific to the decoder it was trained with, or does it represent a general-purpose semantic ordering that any decoder could benefit from? A concrete experiment: take the Stage 3 DeepEncoder V2 (with frozen encoder weights) and pair it with a different decoder architecture (e.g., a dense LLaMA-derived model, or a non-MoE DeepSeek variant) that is then fine-tuned on the document OCR task. Compare performance against: (a) the same decoder trained from scratch with the DeepEncoder V2, and (b) the same decoder trained with the raster-scan DeepEncoder. If DeepEncoder V2's reordering transfers across decoders without substantial degradation, the ordering is a general visual property; if performance drops significantly, the ordering is decoder-specific and the encoder and decoder function as a tightly coupled pair. This has direct practical implications for whether DeepEncoder V2 can be used as a drop-in encoder replacement in existing VLM architectures or requires co-design with the decoder.

Scaling the encoder: does a larger LLM backbone produce better reordering? The paper uses Qwen2-0.5B (500M parameters) as the encoder backbone, chosen partly for parameter parity with CLIP ViT (~300M). An obvious scaling experiment: replace Qwen2-0.5B with a larger LLM backbone (e.g., Qwen2-1.5B or Qwen2-7B) and measure whether the reordering quality scales with encoder capacity. The prediction: larger backbones should produce better reordering (lower reading-order ED) because they have more capacity to model complex layout structures, and this should be most visible on the document types where DeepEncoder V2 currently struggles (newspapers, magazines). The practical question is whether the scaling behavior justifies the additional encoder cost, and whether there is a point of diminishing returns where the decoder becomes the bottleneck regardless of ordering quality. This experiment connects to the broader question of optimal allocation of parameters between encoder and decoder in VLM architectures—a question that the paper's framework makes newly tractable by providing a clear metric (reading-order ED) that isolates encoder contribution from decoder contribution.

Practical Applications and Downstream Use Cases

Production OCR for LLM pretraining data pipelines. The paper's most directly actionable finding for practitioners is that replacing a CLIP-based encoder with an LLM-style encoder incorporating causal queries reduces repetition rate by 33% on user-uploaded images (6.25% → 4.17%, Table 4) and 22% on PDF data (3.69% → 2.88%). For organizations running large-scale document digitization pipelines to generate pretraining data for language models, repetition in OCR output creates cascading quality problems—repetitive text in training data teaches language models to be repetitive, garbled reading order produces incoherent training sequences, and post-processing to detect and filter repetition adds engineering complexity and compute cost. Deploying DeepEncoder V2 in these pipelines directly reduces the downstream cleanup burden while maintaining or improving extraction quality (as measured by the 3.73% OmniDocBench improvement). The architecture is particularly well-suited to this use case because the production metrics (Table 4) demonstrate real-world robustness beyond benchmark performance, and the frozen-encoder Stage 3 training (Section 4.2.3) enables rapid data consumption at deployment scale by eliminating backward passes through the encoder.

Online document understanding services with strict latency and token budgets. The paper positions DeepEncoder V2 at a deliberately low visual token budget (256–1,120 tokens, matching Gemini-3 Pro's upper bound) and demonstrates that this budget achieves 91.09% overall accuracy on OmniDocBench v1.5—outperforming models using 5–7× more tokens (Table 1). For cloud-based document understanding APIs where per-request compute cost directly determines profitability, the token efficiency translates to lower decoder attention costs (quadratic in sequence length) and reduced GPU memory pressure at serving time. A service processing millions of document pages per day at the 1,120 token budget versus a 7,000 token budget saves approximately (7,000/1,120)^2 ≈ 39× in decoder self-attention computation per request—and the paper's results suggest this comes with higher accuracy, not a tradeoff. The multi-crop strategy (Section 3.2.3) provides an additional deployment advantage: the per-crop encoder forward pass has fixed computational cost regardless of original image size, enabling predictable latency and straightforward hardware provisioning.

Document VQA and information extraction where reading order determines answer correctness. Many document understanding tasks require not just extracting text but understanding its structural relationships—"What is the total revenue in the table?" requires knowing which numbers belong to the revenue row, "Summarize the conclusions section" requires identifying where the conclusions begin in a multi-column layout, and "What does footnote 3 refer to?" requires connecting spatially separated elements. The reading-order improvement from DeepEncoder V2 (R-order ED reduced from 0.085 to 0.057, Table 1) directly improves the decoder's ability to answer such questions because the encoder presents information in an order that respects document structure rather than spatial coordinates. For applications where structural understanding is the primary bottleneck (legal document review, financial report extraction, medical record digitization), deploying DeepEncoder V2 could provide accuracy improvements even if the raw text recognition quality is comparable to existing systems. The per-document-type breakdown in Table 3 provides guidance on which document categories benefit most: colorful textbooks, notes, and exam papers show the largest reading-order improvements, while academic papers and research reports (with simpler layouts) show smaller gains—suggesting the architecture is most valuable for document types with complex, non-linear reading patterns.

When to Prefer This Method

The paper explicitly positions DeepEncoder V2 against the standard CLIP-based encoder in DeepSeek-OCR, and the evaluation compares against both pipeline-based and end-to-end models on the OmniDocBench benchmark. Based on the evidence presented, the following decision conditions emerge:

• Prefer DeepEncoder V2 (LLM-style encoder with causal flow queries) when: the primary task involves document reading or layout understanding where spatial order and semantic reading order diverge; the visual token budget is constrained (1,120 tokens or fewer) and efficient token utilization matters; the deployment benefits from fixed per-crop computational cost via multi-crop processing (predictable latency); the document distribution includes a mix of simple and complex layouts, but complex layouts (magazines, newspapers) constitute a minority of requests (since DeepEncoder V2 shows minimal or negative gains on the most complex categories); and the production environment can tolerate the encoder's ~500M additional parameters relative to a standard CLIP ViT (~300M).

• Prefer a standard CLIP-based encoder or a larger-budget VLM when: the document distribution is dominated by newspaper or magazine layouts where DeepEncoder V2's text recognition degrades (Table 3: newspaper Text ED 0.139, worse than DeepSeek-OCR's 0.131) or reading-order improvement is negligible (magazine R-order ED 0.100 → 0.100); the task is general visual reasoning (VQA, captioning, scene understanding) for which the paper provides no evidence of benefit; the deployment has abundant visual token budget (6,000+ tokens) and the priority is maximum raw information extraction rather than token efficiency; or the encoder must be a drop-in CLIP replacement without retraining the decoder (since DeepEncoder V2 requires joint encoder-decoder training in Stage 2 and decoder specialization in Stage 3, and it is unknown whether its reordering generalizes across decoder architectures).

Important caveat on these conditions. The paper evaluates on exactly one task domain (document OCR) and one benchmark (OmniDocBench v1.5). The conditions above apply strictly to document reading; for any other visual task, the paper provides no empirical guidance on whether DeepEncoder V2 helps, hurts, or is neutral. The finding that newspaper text recognition degrades despite the architectural improvements suggests that the causal reordering mechanism has failure modes that are not fully characterized—practitioners deploying on document types not represented in the OmniDocBench categories (legal contracts, engineering diagrams, multilingual layouts with right-to-left scripts) should conduct their own evaluations rather than assuming the OmniDocBench improvements will transfer.