DeepSeek-OCR: Contexts Optical Compression

🎯 Pitch

DeepSeek-OCR achieves 97% OCR precision by using up to 10x fewer vision tokens than text tokens, showing that document pages can be optically compressed into compact representations. Remarkably, it still retains 60% accuracy even at a brutal 20x compression ratio, challenging the assumption that high-resolution tokens are always needed for text-heavy tasks. This flips OCR from an engineering problem into a lens for studying long-context compression limits and memory efficiency in language models.

1. Executive Summary

This paper proposes DeepSeek-OCR as an initial investigation into the feasibility of contexts optical compression — using visual tokens as a compressed, fixed-size representation of text-rich documents — implemented via a novel DeepEncoder architecture (serial SAM-base window attention → 16× convolutional compression → CLIP-large global attention) paired with a DeepSeek3B-MoE-A570M decoder. Evaluated on the Fox and OmniDocBench benchmarks, DeepSeek-OCR achieves ~97% OCR precision at compression ratios under 10× (decoding ~10 text tokens per vision token) and retains ~60% accuracy at 20× compression, while on OmniDocBench it surpasses GOT-OCR2.0 using only 100 vision tokens (vs. 256) and outperforms MinerU2.0 with fewer than 800 vision tokens (vs. ~6,000+) — establishing that optical compression can substitute for orders-of-magnitude more vision tokens only when the compression ratio remains within approximately 10× of the ground-truth text token count.

2. Context and Motivation

The Core Problem: Vision Encoders Produce Too Many Tokens for Long Documents

The fundamental tension this paper addresses is a scaling mismatch in vision-language models (VLMs) : as document images grow larger and more text-dense, existing vision encoders produce linearly — or super-linearly — more vision tokens, which then must be processed by the language model decoder with quadratic attention cost. A standard academic paper page rendered at 1024×1024 resolution and processed by a typical ViT-based encoder might produce ~4,000–7,000 vision tokens (Table 3). For a 50-page document, this balloons to hundreds of thousands of tokens before the language model has even begun generating output text. This token explosion is the direct cause of the long-context bottleneck in LLMs — the inability to process extremely long sequences efficiently because attention scales quadratically with sequence length.

The paper reframes this as a compression problem: if a document image containing 1,000 words of text ground truth can be represented with, say, 100 vision tokens instead of 4,000, the language model's effective context window stretches by a factor of 40× without any architectural change to the LLM itself. The authors state this explicitly:

"Current Large Language Models (LLMs) face significant computational challenges when processing long textual content due to quadratic scaling with sequence length. We explore a potential solution: leveraging visual modality as an efficient compression medium for textual information."

This is not merely about making OCR faster — it is about rethinking the information density of the vision-LLM interface as a first-class design constraint. If vision tokens can serve as a compressed, lossy-but-decodable representation of text, then the vision encoder becomes a compression engine whose compression ratio directly determines how much effective context the downstream LLM can process.

The Unanswered Question: How Many Vision Tokens Does a Document Actually Need?

Prior OCR and document parsing models treated vision token count as an incidental engineering artifact — a consequence of resolution choices and encoder architecture. The paper identifies a critical gap:

"a crucial research question that current models have not addressed is: for a document containing 1000 words, how many vision tokens are at least needed for decoding? This question holds significant importance for research in the principle that 'a picture is worth a thousand words.'"

This question is deceptively simple but has been unexamined. The field knows that models can achieve high OCR accuracy with generous token budgets (thousands of vision tokens per page), but there has been no systematic investigation of the lower bound — the minimum number of vision tokens required to achieve a given decoding accuracy, and how that minimum scales with the document's text complexity. This is the paper's central empirical contribution: measuring the compression ratio boundary at which optical decoding remains viable, and characterizing how precision degrades as that boundary is pushed.

The importance of this question extends beyond OCR. If a document with 1,000 text tokens can be reliably decoded from 100 vision tokens, then an LLM equipped with this encoder could process 10× more content in the same context window — effectively extending its working memory by an order of magnitude. For multi-turn conversations, long-document analysis, and agent systems that accumulate context, this compression directly translates to longer effective memory without architectural changes.

Where Existing Vision Encoders Fall Short

The paper identifies three families of vision encoders used in current open-source VLMs (Section 2.1, Figure 2), and argues that each suffers from deficiencies that make them unsuitable for optical compression research:

1. Dual-tower architectures (e.g., Vary). These use parallel encoders — typically a standard ViT plus a SAM-based encoder — to increase the visual vocabulary parameters for high-resolution processing. While they offer controllable activation memory, they have significant deployment downsides: they require dual image preprocessing (two separate encoding pipelines on the same input) which complicates deployment, and they prevent pipeline parallelism during training because both towers must synchronize before their outputs can be combined. This makes large-scale training with these architectures inefficient.

2. Tile-based methods (e.g., InternVL series). These divide a high-resolution image into smaller tiles (e.g., 448×448), encode each tile independently with a ViT, and concatenate the resulting token sequences. The approach reduces per-tile activation memory (each tile is small), but has three critical flaws:

The native encoder resolution is typically below 512×512, meaning any image larger than this gets fragmented into many tiles. A 4000×3000 image could produce dozens of tiles.
Because each tile produces a fixed number of tokens, the total vision token count scales with image area, not with information content. A large image with sparse text produces just as many tokens as a dense one.
The resulting token sequences are very long. Table 3 shows InternVL2-76B averaging 6,790 vision tokens per page — nearly 7,000 tokens for a single document page, consuming the equivalent of ~5,000 words of LLM context budget just to represent the input image.

3. Adaptive resolution methods (e.g., Qwen2-VL series). These use the NaViT paradigm to process images at their native aspect ratio through patch-based segmentation without tile parallelization. While flexible, they face massive activation memory consumption for large images (the entire image's patches must be processed together in the dense attention layers, causing GPU memory overflow) and require extremely long sequence lengths during training because the packing of variable-length sequences becomes inefficient. Table 3 shows Qwen2.5-VL-7B averaging 3,949 vision tokens per page — still nearly 4,000 tokens for what might be a page with only 600–800 words of text.

The paper's critique is that all three families produce vision token counts that are proportional to image resolution, not to information content. A page with 100 words and a page with 1,000 words, rendered at the same resolution, produce the same number of vision tokens in all these architectures. This is fundamentally wasteful: the tokens carry redundant spatial information rather than compact semantic information about the text.

The Deeper Motivation: Vision as a Memory Compression Medium

Beyond the immediate OCR application, the paper is motivated by a more speculative but potentially transformative idea: using visual rendering as a general-purpose compression mechanism for long-form text in LLMs.

The authors draw an explicit analogy to human memory (Figure 13, Section 5). Human memory for events degrades over time — recent events are "crystal clear" while distant events become "blurry" and eventually "almost gone." The paper proposes simulating this biologically-inspired forgetting curve through progressive optical compression: render dialogue histories, documents, or other long-form text as images at different resolutions depending on their temporal distance, with older content rendered at progressively lower resolutions (and thus represented by fewer vision tokens).

This creates a natural forgetting mechanism where:

Recent context (the last few turns of conversation) is rendered at high resolution and consumes many vision tokens — faithful representation.
Medium-term context (conversations from hours ago) is rendered at medium resolution — compressed but still decodable.
Long-term context (conversations from days or weeks ago) is rendered at low resolution — highly compressed, capturing only the gist that remains decodable.
Very old context is rendered at the minimum resolution — essentially a memory that has faded almost entirely.

The key insight is that this completely sidesteps the quadratic attention problem: instead of storing all historical text as explicit tokens in the LLM's context window, the system stores compressed visual representations that consume far fewer tokens. The LLM "recalls" historical information by decoding these visual memories on demand. The OCR task in this paper serves as the proof-of-concept for the fundamental operation this requires — reliable decoding of text from heavily compressed visual representations.

This is why the compression ratio study (Table 2, Figure 1a) is the paper's centerpiece, not the OmniDocBench leaderboard results. The authors are asking: if we compress text into an image and then severely downsample the image, how much of the original text can we recover? The answer — ~97% at 10× compression, ~60% at 20× — establishes quantitative bounds for what a memory-compression system based on this principle could achieve.

How This Paper Positions Itself

The paper explicitly frames itself as a preliminary proof-of-concept ("initial investigation," "preliminary validation"), not a production-ready system for all OCR use cases. The key differentiators from prior work are:

Against existing OCR models (GOT-OCR2.0, Nougat, OLMOCR, etc.): The paper does not claim to be the most accurate OCR model overall — several larger models (Gemini 2.5-Pro, dots.ocr at 200dpi) achieve lower edit distances on OmniDocBench (Table 3). The claim is instead about efficiency: DeepSeek-OCR achieves competitive or state-of-the-art performance while using 1–2 orders of magnitude fewer vision tokens. With only 100 vision tokens (Small mode), it surpasses GOT-OCR2.0 (256 tokens) on overall edit distance (0.205 vs. 0.280). With 256 tokens (182 valid), it achieves 0.156 overall edit distance, outperforming many models using 15–25× more vision tokens. This is the "doing more with less" thesis.

Against existing vision encoders: DeepEncoder is positioned as the first architecture specifically designed to decouple high-resolution perception from token count. The key architectural innovation is the serial design: a window-attention SAM module processes the full-resolution image with low activation memory, then a 16× convolutional compressor aggressively reduces the token count before the tokens enter the dense global attention CLIP module. This means the expensive global attention operation processes only 1/16th of the tokens that a standard ViT would, while the window-attention module handles the high-resolution visual perception at low cost (window attention scales linearly with image size, not quadratically).

Against the broader VLM field: The paper is positioning optical compression as a new research direction, not just an architectural contribution. The claim is that the VLM community has focused on accuracy at the expense of efficiency — adding more vision tokens to improve performance — while ignoring the possibility that aggressive compression might be viable and would unlock qualitatively different capabilities (ultra-long context, memory systems). The paper argues this is especially relevant because "multimodal systems inherently require an additional vision encoder" — meaning the vision pipeline is already a sunk cost in VLM infrastructure, and using it for compression adds "no overhead" beyond what the system already needs for visual understanding.

The paper is careful to position its results as lower bounds rather than ceilings. The compression ratios achieved (10× near-lossless, 20× with 60% recovery) are from a relatively small decoder (570M activated parameters) with straightforward OCR training. The authors explicitly hypothesize that larger language models, with specialized pretraining, would achieve better decoding:

"It is reasonable to conjecture that LLMs, through specialized pretraining optimization, would demonstrate more natural integration of such capabilities."

This positions DeepSeek-OCR as a demonstration of feasibility rather than a performance-maximized system — the goal is to show the concept works well enough to justify further research, not to solve all OCR problems.

The Practical Deployment Angle

Beyond the research motivation, the paper is also responding to a concrete practical need: large-scale training data generation for LLMs and VLMs. The current paradigm for pretraining multimodal models requires enormous quantities of text extracted from documents (PDFs, web pages, scanned materials). Traditional pipeline approaches (detection → recognition → assembly) are fragile, and existing end-to-end models are too computationally expensive to run at scale.

DeepSeek-OCR is designed to be cheap enough to deploy at massive scale: the authors report that 20 nodes (8× A100-40G each) can generate 33 million pages of training data per day. This is possible specifically because of the low vision token count — fewer tokens means faster inference (both prefill and generation phases benefit, as the authors note in Section 2.1 that "Long vision tokens will slow down both prefill and generation phases of inference"). This positions DeepSeek-OCR as an infrastructure tool for the LLM research community, not just a research artifact.

3. Technical Approach

3.1 Reader Orientation

This paper presents an optical text compression system — a vision-language model that takes a document image as input and outputs the text it contains, but critically, does so using as few vision tokens as possible to represent the image. The system is an end-to-end VLM consisting of a custom vision encoder (DeepEncoder) that aggressively compresses high-resolution document images into a small number of vision tokens, and a compact Mixture-of-Experts language model decoder (DeepSeek3B-MoE-A570M) that "decompresses" those tokens back into text. The problem it solves is the quadratic scaling of LLM attention with sequence length: by representing document pages with orders-of-magnitude fewer tokens than traditional vision encoders produce, the system enables processing dramatically more document content within a fixed LLM context window — essentially turning the vision encoder into a lossy compression engine for text. The "shape" of the solution is serial compression architecture + multi-resolution training + large-scale OCR pretraining, designed not just for good OCR accuracy but specifically optimized for the accuracy-vs-compression-ratio tradeoff curve.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components connected in a linear pipeline:

Preprocessing & Resolution Selection Module: Determines which resolution mode to use (Tiny/Small/Base/Large/Gundam) based on the deployment context or compression target. Images are resized or padded to match the selected native resolution, or tiled in dynamic resolution mode. This module does not learn — it is a fixed geometric transformation.
DeepEncoder (Vision Encoder, ~380M parameters): The core compression engine. It has three sub-components in series:
- SAM-base window attention module (80M): Processes the full-resolution image patches (16×16 pixel patches) using efficient window-based attention. This captures fine-grained spatial and visual details at high resolution without quadratic memory cost.
- 16× convolutional compressor: A 2-layer CNN that downsamples the SAM output tokens by a factor of 16 in each spatial dimension (256× reduction in token count before the global attention stage). This is the aggressive compression step.
- CLIP-large global attention module (300M): Processes the now-compressed token sequence with dense global attention, incorporating pretrained visual-semantic knowledge. Input tokens are now 1/16th the original count.
MoE Language Model Decoder (DeepSeek3B-MoE, 570M activated parameters): A 3-billion-parameter Mixture-of-Experts transformer with 64 routed experts (6 activated per token) plus 2 shared experts. It receives the compressed vision tokens from DeepEncoder concatenated with a text prompt, and autoregressively generates the output text — effectively decompressing the visual representation back into natural language.
Output Post-processing: The generated text is the final output. Depending on the prompt used during inference, this can be plain text (Free OCR), text with layout annotations (bounding boxes and category labels), or specialized formats (HTML tables for charts, SMILES strings for chemical formulas, dictionary-structured geometry).

Information flow: Raw document image → resolution selection (resize/pad/tile) → SAM-base processes patch grid with window attention → 16× convolutional compressor reduces token count → CLIP-large applies global attention to compressed tokens → compressed vision tokens concatenated with text prompt → MoE decoder autoregressively generates text → final output.

3.3 Roadmap for the Deep Dive

First, the DeepEncoder architecture in full detail — the SAM-CLIP serial design, the 16× convolutional compressor, and why this specific design solves the activation memory and token count problems that plague existing encoders. This is the paper's core architectural contribution.
Second, the multi-resolution support system — how a single model is trained to handle resolutions from 512×512 (64 tokens) to dynamic tiling (up to ~2,000 tokens), and the formula for computing valid token counts under padding. This enables the compression ratio experiments.
Third, the MoE decoder — why a 3B MoE architecture with 570M active parameters was chosen, and the formal mapping it learns from compressed latent vision tokens to reconstructed text representations.
Fourth, the data engine — the four categories of training data (OCR 1.0, OCR 2.0, general vision, text-only), the specific data construction pipelines (including the model flywheel for minority languages), and the 70/20/10 data mixture ratio.
Fifth, the two-stage training pipeline — independent DeepEncoder pretraining with a small language model head, followed by full DeepSeek-OCR training with pipeline parallelism (PP=4), including which components are frozen vs. trainable and the specific optimization hyperparameters.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and empirical analysis paper whose core idea is that vision encoders can serve as aggressive text compression engines if designed with low token count and manageable activation memory as first-class objectives, and that compact language models can learn to decode heavily compressed visual representations into text with high fidelity up to approximately 10× compression ratios.

DeepEncoder Architecture: The Serial Compression Design

Problem statement. Traditional vision encoders in VLMs apply a single ViT with dense global attention to the full patch grid of the input image. For a 1024×1024 image with 16×16 patches, this produces 4096 tokens that all attend to each other in every transformer layer — activation memory scales quadratically with token count, and the output token count equals the input patch count. This means high resolution directly implies many vision tokens and high memory consumption.

DeepEncoder solves this by decoupling high-resolution perception from token count through a serial design with an aggressive compression bottleneck in the middle. The key insight is that window attention can handle large token counts efficiently (it scales linearly, not quadratically), while global attention is only needed after the token count has been drastically reduced.

Component 1: SAM-base window attention module (80M parameters). The first stage of DeepEncoder is a Segment Anything Model (SAM) base-sized vision transformer with approximately 80 million parameters, configured with patch size 16. When a 1024×1024 image is input, it is segmented into $1024/16 \times 1024/16 = 4096$ patch tokens. These 4096 tokens are processed through the SAM transformer layers using window-based attention — each token attends only to other tokens within a local spatial window, not to all 4096 tokens globally.

The critical property of window attention is its linear scaling: the computational cost is proportional to the number of tokens multiplied by the window size (a constant), not proportional to the square of the token count. This means the 4096-token input is processed efficiently with manageable activation memory, even though the token count would be prohibitive for global attention.

The paper states the rationale: "Since the first half of encoder is dominated by window attention and only 80M, the activation is acceptable." The choice of SAM-base specifically is motivated by two factors: (1) SAM is pretrained on segmentation tasks that require fine-grained visual perception — its features capture edges, regions, and spatial relationships that matter for text reading; (2) the base size (80M) is small enough that even with window attention, the parameters don't dominate memory.

Component 2: 16× convolutional compressor. Between the SAM and CLIP modules sits a convolutional downsampling module that reduces the token count by a factor of 16 in each spatial dimension. Specifically, the paper uses "a 2-layer convolutional module to perform 16× downsampling of vision tokens."

The detailed specification: each convolutional layer has kernel size 3, stride 2, padding 1, and the channel dimension increases from 256 to 1024 across the two layers. With two stride-2 convolutions, the spatial dimensions are reduced by $2 \times 2 = 4\times$ in each layer, for a total spatial reduction of $4 \times 4 = 16\times$ in each dimension — meaning the token count is reduced by 16×.

For the 1024×1024 input example: the SAM module outputs 4096 tokens. After the compressor, the token count becomes $4096 / 16 = 256$ tokens. This is the aggressive compression step that makes the overall architecture work.

The paper notes that this design is borrowed from Vary, but the critical difference is that in DeepEncoder the convolution sits between the two attention modules rather than being applied to the output of a parallel encoder. This serial placement is what enables the activation memory to remain manageable.

Why 16× specifically? The paper does not ablate this choice, but the design logic is clear: 16× compression brings a 1024×1024 image from 4096 tokens to 256 tokens — a number that is comfortably within the range where CLIP-large's global attention is efficient. A smaller compression factor (e.g., 4× → 1024 tokens) would still be costly for global attention. A larger factor (e.g., 64× → 64 tokens) would risk losing too much spatial information for the downstream text decoding task. The 16× factor is a design sweet spot informed by the target image resolutions and decoder capabilities.

Component 3: CLIP-large global attention module (300M parameters). The final stage uses a CLIP-large vision transformer with approximately 300 million parameters. Critically, the first patch embedding layer of CLIP is removed — "since its input is no longer images but output tokens from the previous pipeline." The compressed 256 tokens (now with 1024 channels from the convolution output) enter the CLIP transformer directly and are processed with dense global attention — every token attends to every other token.

This is where the pretrained visual-semantic knowledge from CLIP is leveraged. CLIP-large was pretrained on 400 million image-text pairs using contrastive learning, giving it representations that align visual features with linguistic concepts. By processing the compressed tokens through CLIP, the encoder benefits from this pretraining while only paying the quadratic attention cost on 256 tokens (not 4096). The CLIP module "understands" what the SAM module "saw" — SAM provides the fine-grained visual features, the compressor reduces them, and CLIP contextualizes them with semantic knowledge.

Total parameter count. DeepEncoder is approximately 380M parameters: 80M (SAM-base) + 300M (CLIP-large). The convolutional compressor adds a small number of parameters (two convolutional layers with channel expansion) but this is negligible relative to the transformer components.

The activation memory property. The paper emphasizes activation memory as the key design constraint. For the 1024×1024 example:

SAM stage: 4096 tokens × window attention = linear scaling with modest constant → manageable.
Compressor stage: reduces 4096 → 256 tokens (a 93.75% reduction).
CLIP stage: 256 tokens × global attention = $256^2 = 65,536$ attention entries per layer → very manageable.

If the entire 4096-token sequence were processed with global attention (as in a standard ViT), the attention matrix would have $4096^2 = 16.8$ million entries — 256× larger, and likely exceeding GPU memory at typical batch sizes.

Why this design over alternatives? The paper explicitly contrasts DeepEncoder with three existing architectures (Figure 2, Section 2.1):

vs. dual-tower (Vary): Parallel SAM + ViT requires synchronizing two separate encoding pipelines, preventing efficient pipeline parallelism during training. DeepEncoder's serial design places SAM and CLIP in sequence, naturally fitting into a pipeline-parallel training setup where SAM is placed on one GPU stage and CLIP on another.
vs. tile-based (InternVL): Tile-based methods produce vision token counts proportional to image area. A 4000×3000 image might require dozens of 448×448 tiles, each producing 256+ tokens, yielding thousands of output tokens. DeepEncoder produces a fixed token count regardless of image content (256 tokens for Base mode, 100 for Small mode, etc.) because the compression factor is applied uniformly. The token count is determined by the target resolution mode, not the image size.
vs. adaptive resolution (Qwen2-VL): NaViT-style encoding processes all patches with global attention, causing massive activation memory for large images. DeepEncoder restricts global attention to only the post-compression tokens, decoupling perception resolution from attention cost.

Multi-Resolution Support: How One Model Handles 64 to 2,000+ Tokens

Why multi-resolution matters for compression research. The paper's central experiment (Table 2, Figure 1a) requires testing how OCR accuracy varies as the number of vision tokens changes. A document with 1,000 words of text — can it be decoded from 64 vision tokens? From 100? From 256? Answering this requires a single model that can operate at multiple token counts, because comparing different models at different token counts would confound model quality with compression ratio.

The paper achieves this through dynamic interpolation of positional encodings and simultaneous multi-resolution training. Rather than training separate models for each resolution, the DeepEncoder is trained on a mixture of resolution modes, with positional encodings interpolated to match the input resolution at each training step.

Native resolution modes. The paper defines four fixed resolution modes, each corresponding to a specific input image size and output token count:

Mode	Native Resolution	Vision Tokens	Processing
Tiny	512×512	64	Resize
Small	640×640	100	Resize
Base	1024×1024	256	Padding
Large	1280×1280	400	Padding

The token counts follow from the architecture: for a $W \times H$ input with patch size 16, the SAM module produces $(W/16) \times (H/16)$ patches. After the 16× compressor, the token count becomes $((W/16) \times (H/16)) / 16$ . For simplicity, the paper reports the token count as the output of the entire DeepEncoder (including the compressor).

Resize vs. padding strategy. Tiny and Small modes use direct resizing — the original image is stretched or compressed to exactly 512×512 or 640×640. This means the entire image area contributes to the vision tokens, but the aspect ratio is distorted. For small resolutions where token count is very limited, preserving the full image content is prioritized over aspect ratio fidelity.

Base and Large modes use padding — the original image is padded with blank pixels to reach 1024×1024 or 1280×1280 while preserving its original aspect ratio. This means some vision tokens correspond to padding regions rather than image content. The number of "valid" (non-padding) vision tokens is:

$N_{\text{valid}} = \lceil N_{\text{actual}} \times [1 - ((\max(w, h) - \min(w, h)) / \max(w, h))] \rceil$

where $w$ and $h$ are the width and height of the original input image, and $N_{\text{actual}}$ is the total number of vision tokens output by DeepEncoder for that resolution mode (256 for Base, 400 for Large).

What it computes: Given an image with aspect ratio $w:h$ , some fraction of the padded canvas is blank. The term $(\max(w,h) - \min(w,h)) / \max(w,h)$ measures the aspect ratio distortion — for a square image it is 0 (no wasted tokens); for a very elongated image it approaches 1 (most tokens are padding). The valid token count subtracts this waste. For example, a 1024×768 image in Base mode (1024×1024 canvas): $N_{\text{valid}} = 256 \times [1 - ((1024-768)/1024)] = 256 \times [1 - 0.25] = 256 \times 0.75 = 192$ valid tokens out of 256.

Why this form: The ceiling function $\lceil \cdot \rceil$ ensures an integer token count. The formula empirically captures how much of the encoder's output corresponds to actual image content vs. padding, which matters for understanding the effective compression ratio in experiments. The paper reports both actual and valid token counts in Table 3 (e.g., "256(182)" for Base mode on OmniDocBench, meaning 256 total tokens of which 182 are valid on average across test images).

Dynamic resolution (Gundam mode). For production use cases with very high-resolution inputs (newspapers, dense PDF pages), the paper introduces a tiled dynamic resolution mode called Gundam. This mode composes two native resolutions:

$n$ tiles of 640×640 each (local views, using Small mode's resolution)
One global view of 1024×1024 (Base mode resolution)

The tiling method follows InternVL2.0: the high-resolution image is divided into $n$ overlapping 640×640 tiles, each processed independently through the SAM and compressor stages. The global view is a downsampled version of the full image at 1024×1024. Both local and global views are fed through the CLIP module jointly.

The output token count is:

$N_{\text{Gundam}} = n \times 100 + 256$

where $n$ is the number of tiles (controlled to be in the range 2 to 9). The 100 comes from Small mode (640×640 → 100 tokens), and the 256 from Base mode (1024×1024 → 256 tokens after compression).

For images where both width and height are smaller than 640, $n$ is set to 0, and Gundam mode degrades to Base mode (256 tokens). This ensures that small images don't get unnecessarily tiled.

Why tiling helps further reduce activation memory. Tiling is itself a form of secondary window attention: each 640×640 tile is processed independently by the SAM module, so the per-tile activation memory is small. The global view (1024×1024) provides cross-tile context in the CLIP stage. Because the native tile resolution (640) is relatively large compared to typical tiling approaches (e.g., InternVL uses 448×448 tiles), fewer tiles are needed: "images won't be fragmented too much under dynamic resolution (the number of tiles is controlled within the range of 2 to 9)."

Gundam-Master mode (extended training). An even larger variant uses 1024×1024 local views (Base mode resolution per tile, 256 tokens each) plus a 1280×1280 global view (Large mode, 400 tokens), yielding:

$N_{\text{Gundam-M}} = n \times 256 + 400$

The paper notes this mode is not trained jointly with the others because "Gundam-master's resolution is too large and training it together would slow down the overall training speed." Instead, it is obtained through continued training on a pre-trained DeepSeek-OCR model using 6 million sampled data examples.

How positional encoding interpolation works. Since the model is trained on images of varying sizes (512×512, 640×640, 1024×1024, 1280×1280, and tiled composites), the positional encodings must adapt. The paper states the model uses "dynamic interpolation of positional encodings" — the pretrained positional embeddings (which were likely trained at a fixed resolution, such as 1024×1024 for SAM and CLIP) are interpolated (stretched or compressed) to match the current input resolution. This is a standard technique in vision transformers for handling variable input sizes, and the paper leverages it without introducing novel mechanisms.

Training with multiple modes simultaneously. The key to making a single model work across all resolutions is training them jointly. During training, each batch contains a mixture of images at different resolution modes, and the model learns to handle all of them. The paper does not specify the exact mixing proportions for the different resolution modes, but the fact that all four native modes plus Gundam mode are trained together means the CLIP module learns to process variable-length token sequences (64, 100, 256, 400, or dynamic counts from tiling) from the same weights.

The MoE Decoder: Decompressing Vision Tokens Back to Text

Architecture choice. The decoder is DeepSeek3B-MoE, a 3-billion-parameter Mixture-of-Experts language model based on the DeepSeekMoE architecture. During inference, it activates 6 out of 64 routed experts plus 2 shared experts, for approximately 570 million activated parameters.

The rationale for this choice is explicitly about the inference efficiency vs. capacity tradeoff: "The 3B DeepSeekMoE is very suitable for domain-centric (OCR for us) VLM research, as it obtains the expressive capability of a 3B model while enjoying the inference efficiency of a 500M small model."

In MoE architectures, "activated parameters" refers to the subset of the model actually used for a given token — the router selects which experts to engage, and only those experts' computations are performed. The total parameter count (3B) determines the model's capacity to store knowledge, while the activated parameter count (~570M) determines the inference cost per token. This is ideal for OCR where the model needs broad knowledge (many languages, diverse document formats, chart types, chemical notation) but inference cost must be low for large-scale deployment.

Input and output specification. The decoder receives two inputs concatenated into a single sequence: the compressed vision tokens from DeepEncoder (a sequence of $n$ vectors, each in $\mathbb{R}^{d_{\text{latent}}}$ ) and a text prompt tokenized by the standard DeepSeek tokenizer (vocabulary size approximately 129k tokens). It autoregressively generates text tokens until an end-of-sequence condition is met.

The formal mapping. The paper formalizes the decoder's role as learning a non-linear mapping from compressed visual latent space to text token space:

$f_{\text{dec}} : \mathbb{R}^{n \times d_{\text{latent}}} \to \mathbb{R}^{N \times d_{\text{text}}}; \quad \hat{\mathbf{X}} = f_{\text{dec}}(\mathbf{Z}) \quad \text{where } n \leq N$

where $\mathbf{Z} \in \mathbb{R}^{n \times d_{\text{latent}}}$ is the matrix of compressed latent (vision) tokens from DeepEncoder, $\hat{\mathbf{X}} \in \mathbb{R}^{N \times d_{\text{text}}}$ is the reconstructed text representation in the output embedding space, $n$ is the number of vision tokens (e.g., 64, 100, 256), $N$ is the number of output text tokens (which can be much larger than $n$ ), $d_{\text{latent}}$ is the hidden dimension of DeepEncoder's output (determined by the CLIP-large architecture), and $d_{\text{text}}$ is the hidden dimension of the language model.

What it computes: Given $n$ compressed vision tokens (each a high-dimensional vector encoding visual information about a region of the document), the decoder learns to generate a sequence of $N$ text tokens that represent the text content of the original document. The inequality $n \leq N$ is the compression relationship — the model reconstructs more text tokens than the number of vision tokens it received, achieving the compression ratio $N/n$ .

Why this form: The mapping from $\mathbb{R}^{n \times d}$ to $\mathbb{R}^{N \times d}$ where $n \ll N$ is precisely the decompression operation. Standard VLMs typically have $n \gg N$ — many more vision tokens than text tokens — because they encode each image patch separately. Here the relationship is inverted: the vision tokens are a bottleneck that the language model must learn to "read" efficiently. The notation emphasizes that this is a learned transformation, not a deterministic decoding, and that the decoder must infer the text content from an information-theoretically compressed representation.

The conjecture about larger LLMs. The paper speculates that the decompression capability demonstrated by the 570M-active-parameter decoder represents a lower bound, and that larger language models with specialized pretraining would perform even better: "It is reasonable to conjecture that LLMs, through specialized pretraining optimization, would demonstrate more natural integration of such capabilities." This is important because it positions DeepSeek-OCR as a feasibility demonstration, not a ceiling on what optical compression can achieve.

Prompt-based output control. The decoder's behavior is controlled by the text prompt appended to the vision tokens. Different prompts elicit different output formats:

"<image>\nFree OCR.": plain text extraction without layout information
"<image>\n<|grounding|>Convert the document to markdown.": text extraction with layout annotations (bounding boxes and category labels)
"<image>\nParse the figure.": specialized parsing for charts, chemical formulas, or geometric figures (the "deep parsing" capability)
"<image>\nLocate <|ref|>object<|/ref|> in the image.": visual grounding

This prompt-based control means a single model handles diverse output tasks without architectural changes, with the prompt routing the decoder's generation behavior.

Data Engine: Constructing the Training Mixture

Overview and motivation. The training data is organized into four categories with a specific mixing ratio: OCR data (70%), general vision data (20%), and text-only data (10%). This mixture is designed to produce a model that excels at document parsing and text extraction while retaining general visual understanding and language capabilities. The paper explicitly states: "DeepSeek-OCR is not a general VLM model, and this portion of data accounts for only 20% of the total data. We introduce such type of data mainly to preserve the general vision interface."

OCR 1.0 Data: Traditional Text Recognition

This is the largest data category and the primary training signal for the compression-decompression task. It consists of two sub-types:

Document OCR data (PDF-based): The paper collects 30 million pages of diverse PDF data from the internet covering approximately 100 languages, with Chinese and English comprising roughly 25 million pages and other languages 5 million. The data is annotated at two quality levels:

Coarse annotations (bulk of the data): Text content extracted directly using PyMuPDF (fitz), a PDF parsing library that can extract text from born-digital PDFs. This is fast and scalable but captures only text content without spatial layout information. The purpose is to teach the model the fundamental optical character recognition task — mapping visual features to character sequences — across many languages.
Fine annotations (2M Chinese + 2M English pages): High-quality annotations with interleaved layout and recognition information. The pipeline is: (1) detect document layout elements using PP-DocLayout (a pretrained layout detection model), (2) recognize text within each layout element using MinerU and GOT-OCR2.0 (advanced OCR models), (3) format the output as an interleaved sequence where each paragraph of text is preceded by its bounding box coordinates and element category label. All coordinates are normalized to 1000 bins (i.e., image dimensions are scaled to [0, 1000] range). Figure 5 shows an example of this format.
Minority language flywheel: For non-Chinese/English languages, the layout detection model generalizes somewhat, but the OCR recognition models (MinerU, GOT-OCR2.0) were not trained on these languages. To solve this, the authors create a self-training loop: (1) use PyMuPDF to extract text from small image patches cropped from the PDF pages, creating training pairs of (small image patch, ground truth text), (2) fine-tune GOT-OCR2.0 on these patch-level pairs to create a language-specific OCR model, (3) use the fine-tuned model to label the full pages after layout detection, (4) iterate. This "model flywheel" produces 600K training samples for minority languages without requiring manual annotation.

Word data (non-PDF): 3 million Word document pages are collected and converted to images. The ground truth is extracted directly from the document content (not OCR-inferred), creating high-quality image-text pairs without layout information. This data particularly benefits formula and HTML-formatted table recognition because Word documents accurately represent mathematical notation and table structures.

Natural scene OCR data: 10 million Chinese and 10 million English images from LAION and Wukong datasets, labeled using PaddleOCR, a production OCR system. These cover text-in-the-wild scenarios (signs, product labels, street scenes) distinct from document images.

Open-source supplements: Additional data from existing open-source OCR datasets (OLMoCR, Vary-tiny) are included as supplementary training data.

OCR 2.0 Data: Parsing Structured Visual Content

This category extends beyond text recognition to parsing specialized visual formats, following the OCR 2.0 taxonomy from GOT-OCR2.0:

Chart data (10M images): Charts are rendered programmatically using pyecharts and matplotlib libraries, covering line charts, bar charts, pie charts, and composite charts. The ground truth is HTML table format (not the dictionary format used in OneChart). The authors argue HTML tables "save a certain amount of tokens" compared to dictionary representations. The task is image-to-HTML-table conversion — the model must infer the data values from the visual chart representation. Figure 6(a) shows an example.

Chemical formula data (5M images): Chemical structures from PubChem are converted to SMILES string representations and rendered as images using RDKit (a cheminformatics library). Each training example pairs a rendered molecular structure image with its SMILES string ground truth. This teaches the model to parse 2D molecular diagrams into machine-readable chemical notation.

Plane geometry data (1M images): Geometric figures are generated following the Slow Perception approach: each line segment is modeled with a "perception-ruler" size of 4 (encoding each segment as a sequence of 4 tokens representing its endpoints and properties). The ground truth is a dictionary format with keys for line segments, endpoint coordinates, line segment types, etc., designed for readability. A geometric translation-invariant data augmentation is applied: the same geometric figure is translated (shifted) within the image canvas, while the ground truth remains referenced to a centered coordinate system. This ensures the model learns geometric relationships independent of absolute position. Figure 6(b) shows an example.

General Vision Data (20% of total):

Following the data generation approach of DeepSeek-VL2, the paper includes training data for caption generation, object detection, and visual grounding tasks. This preserves the general visual understanding capabilities inherited from CLIP's pretraining, ensuring the model can describe images, locate objects, and answer visual questions when prompted appropriately. The paper is explicit that this is a secondary capability — DeepSeek-OCR is not designed as a general VLM, and this data exists "mainly to preserve the general vision interface, so that researchers interested in our model and general vision task can conveniently advance their work in the future."

Text-only Data (10% of total):

10% of training data is in-house text-only pretraining data, processed to a maximum length of 8192 tokens. This serves two purposes: (1) preventing catastrophic forgetting of language capabilities during vision-heavy training, and (2) maintaining the decoder's general language modeling ability so it can produce fluent, well-structured output text. The 8192 token length is also the sequence length used for DeepSeek-OCR training overall.

Why the 70/20/10 split? The paper does not ablate this ratio, but the design logic is apparent: OCR data dominates because it directly teaches the compression-decompression task; general vision data provides enough signal to maintain visual understanding without overwhelming the OCR specialization; text-only data provides a small but consistent language regularization signal. The 10% allocation to text-only data is notably small — this is possible because the OCR task itself involves generating text, so language capability is partially maintained through the OCR training examples.

Training Pipelines: Two-Stage Optimization

Overview. The training follows a two-stage process: (1) pretrain DeepEncoder independently using a compact language model as a training auxiliary, then (2) train the full DeepSeek-OCR model with the frozen SAM+compressor and trainable CLIP+decoder. A third optional stage (Gundam-Master continued training) exists but uses identical protocol to stage 2 with different data.

Stage 1: Training DeepEncoder Independently

Motivation. DeepEncoder needs to learn to extract text-relevant visual features before being connected to the full decoder. Training the full model end-to-end from scratch would be inefficient because the encoder would receive weak gradient signal through the randomly initialized decoder. Pretraining the encoder with a simpler objective allows it to converge faster when the full model is trained.

Setup. Following the Vary approach, a compact language model (OPT-IML, a 1.3B instruction-tuned model) is attached to DeepEncoder's output. The combined model is trained with the standard next-token prediction objective: given the compressed vision tokens and a prompt, generate the ground truth text autoregressively.

Training data. All OCR 1.0 and OCR 2.0 data described in Section 3.4, plus 100 million general vision images sampled from the LAION dataset. The inclusion of general vision data at this stage ensures the encoder doesn't over-specialize to text features at the expense of general visual features that CLIP's pretraining provides.

Hyperparameters. Training runs for 2 epochs with batch size 1280, using the AdamW optimizer with a cosine annealing learning rate scheduler and learning rate $5 \times 10^{-5}$ . The training sequence length is 4096 tokens. (Note: the paper does not specify the number of GPUs used for this stage, the warmup steps, weight decay, or gradient clipping values.)

What this stage produces. After 2 epochs, DeepEncoder's weights are updated to produce vision tokens that a small language model can decode into accurate text. The SAM, compressor, and CLIP components are all trained jointly in this stage. After completion, the compact language model is discarded — its only purpose was to provide a training signal.

Stage 2: Training DeepSeek-OCR (Full Model)

Pipeline parallelism configuration. The full training uses the HAI-LLM training platform with pipeline parallelism (PP) dividing the model into 4 stages:

PP0: SAM-base module + 16× convolutional compressor. These are treated as the "vision tokenizer" and have their parameters frozen during this stage. The paper explicitly states: "For DeepEncoder, we treat SAM and the compressor as the vision tokenizer, place them in PP0 and freeze their parameters."
PP1: CLIP-large module. This is treated as an "input embedding layer" for the language model — its parameters remain unfrozen and are updated during training. The reasoning is that the CLIP module needs to adapt its global attention processing to the compressed token sequences produced by the frozen SAM+compressor, and to learn which visual-semantic features are most relevant for text decoding.
PP2: First 6 layers of DeepSeek3B-MoE (out of 12 total layers).
PP3: Last 6 layers of DeepSeek3B-MoE.

Why freeze SAM and the compressor? The SAM module does not need further training because its job — extracting fine-grained visual features from high-resolution patches — is well-served by its pretrained weights and the Stage 1 training. The compressor's convolutional layers are simple (2 layers, kernel 3, stride 2) and their downsampling function is primarily geometric rather than learned. Freezing these components reduces the memory footprint during training and prevents the aggressive compression from being "undone" by the training signal, which might push the compressor to preserve more information (and thus more tokens) than desired.

Why keep CLIP trainable? CLIP is the component that must learn to extract text-decodable information from the compressed token sequence. Its pretrained weights encode general visual-semantic alignments, but the specific task of mapping compressed SAM features to features useful for text generation requires adaptation. Making CLIP trainable allows the gradient signal from the language modeling loss to shape which visual features are preserved and emphasized.

Training infrastructure and scale. 20 nodes, each with 8 A100-40G GPUs (160 GPUs total). Data parallelism is 40 (meaning 40 independent data shards processed in parallel), with a global batch size of 640. The optimizer is AdamW with a step-based learning rate scheduler and an initial learning rate of $3 \times 10^{-5}$ . (The paper does not specify the step decay schedule, weight decay, warmup steps, or gradient clipping.)

Training speed. For text-only data, training processes 90 billion tokens per day. For multimodal data (images + text), training processes 70 billion tokens per day. The slower speed for multimodal data reflects the additional computation of DeepEncoder's forward pass and the variable image sizes in the batch.

Sequence length. The training sequence length is 8192 tokens, matching the text-only data preprocessing. This is also the maximum sequence length the model supports at inference.

Stage 3 (Optional): Gundam-Master Continued Training

After the main DeepSeek-OCR model is trained, the Gundam-Master mode (1024×1024 local tiles + 1280×1280 global view) is obtained through continued training on 6 million sampled data examples. The training protocol is identical to Stage 2 — the paper explicitly says "the training protocol is identical to other modes, we omit the detailed description hereafter." The only difference is the resolution configuration and the data used.

The separation from the main training is motivated by load balancing: "Gundam-master's resolution is too large and training it together would slow down the overall training speed." Including very high-resolution examples in the main training batch would create significant variance in per-step computation time, as GPUs processing Gundam-Master examples would take much longer than those processing Tiny or Small mode examples, causing other GPUs to idle.

Inference setup. At inference, the model uses the prompt to control output format (as described in the decoder section). The paper does not specify inference hyperparameters such as temperature, top-p, top-k, beam search settings, or maximum generation length. The compression experiments (Table 2) use the prompt "<image>\nFree OCR." to extract plain text without layout annotations.

Summary of Design Choices and Their Justifications

Serial SAM → Compressor → CLIP design over parallel or single-encoder designs: decouples high-resolution perception (window attention) from token count, ensuring global attention operates on only 1/16th of the original patch count. This is the single most important architectural decision — without it, either the model could not handle high resolutions (activation memory overflow) or would produce too many tokens (defeating the compression objective).
16× convolutional compressor over learnable pooling or attention-based downsampling: simple, parameter-efficient, and provides a hard information bottleneck that forces the CLIP module to work with aggressively compressed representations. The convolution kernel (3×3, stride 2) is a local operation that preserves spatial structure while reducing resolution, unlike global pooling which would discard all spatial information.
Frozen SAM + compressor, trainable CLIP in Stage 2: the SAM performs generic visual feature extraction (well-served by pretraining), while CLIP must adapt its semantic processing to the specific demands of text decoding from compressed features. The compressor's simplicity means it doesn't benefit from further training.
Multi-resolution joint training over separate models per resolution: enables a single model to serve as the testbed for compression ratio experiments, eliminating model quality as a confounding variable when comparing performance across token counts. Dynamic positional encoding interpolation makes this technically feasible.
MoE decoder with 570M active parameters over dense decoder of similar active size: provides 3B parameter capacity (beneficial for multilingual, multi-format OCR knowledge) at 570M inference cost (critical for large-scale data generation at 200k+ pages per day).
70/20/10 data mixture over OCR-only training: the OCR-dominant mixture ensures strong task performance, while the 20% vision data preserves the general visual interface (important for the model's utility as a research platform) and the 10% text data prevents language degradation.
Flywheel approach for minority language data over manual annotation or pure synthetic data: leverages the generalization of layout models and the scalability of self-training to create training data for ~100 languages without human labeling. The patch-level GOT-OCR2.0 fine-tuning creates a language-specific recognizer that then labels full documents.
Coarse + fine annotation strategy over uniformly high-quality annotations: coarse annotations (fast PyMuPDF extraction) provide volume for basic OCR learning across many languages; fine annotations (layout + recognition models) provide quality for the most important languages (Chinese, English) and the layout-aware parsing capability. This balances training data scale with annotation quality.
Stage-wise training with discarded auxiliary LM over end-to-end training from scratch: allows DeepEncoder to converge to text-relevant features before being connected to the full MoE decoder. The auxiliary LM provides a strong training signal without the computational cost of training the full MoE decoder in Stage 1.

4. Key Insights and Innovations

Innovation 1: Reframing the Vision-LM Interface as an Information-Theoretic Compression Bottleneck

The paper's most fundamental contribution is conceptual, not architectural: it reframes the vision encoder in a VLM not as a perceptual front-end whose job is to faithfully represent the input image, but as a lossy compression engine whose output should be the minimum number of tokens needed by the language model to reconstruct the text content. This is a genuine inversion of the dominant VLM design philosophy.

What the field assumed before this work. The standard approach in VLMs — across all three encoder families critiqued in Figure 2 — has been that vision tokens should faithfully encode the visual signal at a granularity sufficient for the language model to answer arbitrary questions about the image. ViT-based encoders produce one token per patch (typically 14×14 or 16×16 pixels), yielding token counts that are proportional to image area and unrelated to the information content a downstream task actually requires. A blank page and a dense text page at the same resolution produce identical token counts. The field has largely accepted this as an unavoidable cost of visual perception and focused on making the downstream processing (the LLM) more efficient through techniques like flash attention, KV-cache compression, and sparse attention — all of which address the consequences of too many tokens rather than the cause.

The paper identifies this assumption explicitly:

"a crucial research question that current models have not addressed is: for a document containing 1000 words, how many vision tokens are at least needed for decoding?"

What makes this reframing distinctive. The paper treats the number of vision tokens as a first-class design variable rather than an incidental artifact of resolution choices. This shifts the optimization objective from "maximize accuracy at a given resolution" to "maximize accuracy at a given token count" — or equivalently, "find the compression ratio boundary where decoding remains viable." The compression ratio (text tokens decoded per vision token consumed) becomes the primary metric of interest, not just OCR accuracy.

This reframing unlocks a completely different way of thinking about what a vision encoder is for. In the paper's framework, the encoder's job is to produce a compressed latent representation Z with the property that a language model can decode it into text — essentially a learned image → text compression codec where the compression ratio is tunable via the encoder's output token count. This connects document understanding to classical rate-distortion theory, where the fundamental question is the tradeoff between compression (fewer bits/tokens) and reconstruction quality (OCR accuracy).

Evidence and significance. The compression ratio experiments in Table 2 and Figure 1(a) are the direct empirical manifestation of this reframing. The key finding — ~97% precision at compression ratios under 10×, ~60% at 20× — is not merely a performance number but a characterization of the information-theoretic limits of the learned compression scheme. These numbers establish quantitative bounds on what optical compression can achieve with current architectures, and provide a benchmark for future work that approaches the problem from this reframed perspective.

Why this matters beyond OCR. The reframing opens up entirely new research directions that were invisible under the old framing. If vision tokens can serve as compressed text representations, then optical rendering becomes a general-purpose mechanism for extending LLM context windows: render historical text as images, encode with a compression-optimized encoder, and decode on demand. The paper's memory-forgetting analogy (Figure 13, Section 5) only makes sense under this reframing — it treats vision tokens as a storage medium for text, not just a perceptual representation. This is a fundamentally different relationship between vision and language modalities than what any prior VLM work has proposed.

Innovation 2: The Serial Window-Attention → Compression → Global-Attention Architecture as a Principle for Decoupling Resolution from Token Count

While the paper's architectural contribution (DeepEncoder) is a concrete design, the deeper innovation is a design principle: that high-resolution visual perception and dense semantic processing should be architecturally separated and connected through an aggressive compression bottleneck, rather than being performed by the same transformer layers on the same token count. This principle directly contradicts how standard ViTs and most VLM encoders operate.

What the field did before. All three encoder families in Figure 2 apply global attention to the full (or nearly-full) patch token count at some point in their processing. Tile-based methods (InternVL) apply global attention within each tile, then concatenate all tiles — effectively applying global attention to a number of tokens that scales with image area. Adaptive resolution methods (Qwen2-VL, NaViT) apply global attention to the entire patch sequence, which for large images means global attention over thousands of tokens. Dual-tower methods (Vary) run two parallel global-attention encoders. In all cases, the expensive global attention operation processes a token count that is proportional to the input image resolution — higher resolution means more tokens in global attention, which means quadratically more computation.

What makes DeepEncoder's principle distinctive. The serial design with a 16× compressor interposed between the window-attention and global-attention modules means that the token count entering global attention is decoupled from the input resolution. The SAM module can process a 4096-token patch grid (from a 1024×1024 image) with efficient window attention, and the CLIP module processes only 256 tokens (after 16× compression). If the input resolution were doubled, the SAM module would process 16,384 tokens with window attention (linear scaling), and the CLIP module would process 1,024 tokens (still manageable) — not 16,384.

This is not simply an engineering optimization. It is an architectural hypothesis about where global attention is necessary vs. where local processing suffices. The hypothesis is that the early stages of visual processing — edge detection, texture analysis, character-level feature extraction — can be performed with purely local operations (window attention), while only the later stages that require integrating information across the entire image (reading order, document structure, cross-region context) need global attention. The compressor sits at the boundary between these two regimes.

The paper's evidence for this hypothesis is indirect but compelling: the architecture works — producing state-of-the-art OCR accuracy with dramatically fewer vision tokens than competitors — and no ablation study contradicts it. More tellingly, the design is motivated by a clear diagnosis of why existing encoders fail (Section 2.1, Figure 2), and each design choice addresses a specific diagnosed failure mode:

Window attention in the first stage addresses the activation memory problem of Qwen2-VL-style encoders.
The compressor addresses the token count problem of InternVL-style encoders.
The serial (not parallel) design addresses the pipeline parallelism problem of Vary-style dual-tower encoders.

Comparison to prior attention patterns. The idea of combining local and global attention is not novel — it appears in architectures like Swin Transformer (shifted window attention) and ViTDet (window attention with global propagation). What is novel is the aggressive compression ratio at the boundary: 16× spatial reduction (256× in token count) is far more aggressive than the 2× or 4× reductions typical in hierarchical vision transformers (e.g., Swin's patch merging). The paper's bet is that for document OCR specifically, this extreme compression is viable because text is highly redundant — neighboring pixels within a character convey the same information — and the language model's decoding capability can compensate for lost visual detail.

This bet pays off empirically (Tables 2, 3) and represents a genuinely new point in the design space: extreme compression ratios are viable when the downstream decoder is a powerful language model, which prior work did not explore because it treated vision token count as a given rather than a variable to optimize.

Innovation 3: The Compression Ratio as a Quantitative, Task-Specific Lower Bound on Vision Token Requirements

The paper's third innovation is empirical and diagnostic: it provides the first systematic measurement of the minimum viable compression ratio for optical text decoding, establishing that documents can be reliably decoded at ~10× compression and partially decoded at ~20×, with sharp degradation beyond that. This is not just a performance result — it is a characterization of a fundamental limit that was previously unknown and unmeasured.

What was unknown before. Prior OCR and document parsing papers reported accuracy at whatever token count their architecture produced by default. GOT-OCR2.0 uses 256 tokens per page; that was a design choice, not a finding. InternVL2-76B uses 6,790 tokens; that was a consequence of its tiling strategy, not an optimized number. No prior work asked: "could InternVL2-76B achieve similar accuracy with 500 tokens?" or "is 256 tokens near the lower bound for GOT-OCR2.0, or could it work with 64?" The field had no empirical characterization of the accuracy-vs-token-count tradeoff curve, and therefore no basis for deciding how many vision tokens a document parser should use.

What makes these measurements distinctive. The paper's multi-resolution training design (Section 3.2.2) — where a single DeepEncoder is trained on 64, 100, 256, 400, and dynamic token counts simultaneously — enables a controlled measurement of the compression-accuracy tradeoff. Because the same model weights serve all token counts, differences in accuracy across modes can be attributed to the compression ratio itself, not to differences in model architecture, training data, or optimization. This is methodologically important: prior comparisons between models with different token counts (e.g., GOT-OCR2.0 at 256 vs. InternVL2-76B at 6,790) confound token count with model quality, making it impossible to isolate the effect of compression.

The Fox benchmark results in Table 2 reveal several non-obvious patterns:

Near-lossless decoding at <10× compression. At 6.7–10.6× compression (100 vision tokens for 600–1100 text tokens), precision ranges from 91.5% to 98.5%. This means a document with 1,000 words of text can be reconstructed almost perfectly from just 100 vision tokens — equivalent to representing each word with approximately 0.1 vision tokens. This is a striking empirical finding: the information density achievable with learned compression vastly exceeds what pixel-level encoding would suggest (a 640×640 image contains 409,600 pixels; 100 vectors of ~1024 dimensions each encode this into ~102,400 floating-point numbers, a 4× reduction, but the token count reduction is what matters for LLM context).
Graceful degradation at 10–20× compression. Between 10× and 20× compression, precision degrades smoothly from ~90% to ~60% without catastrophic collapse. This suggests the compression is not brittle — the decoder can still extract partial information even when the visual representation is heavily degraded.
Resolution, not just token count, matters. The paper notes that precision decline beyond 10× may be partly due to "long texts becoming blurred at 512×512 or 640×640 resolution" — a physical limitation of rendering text at low resolution, not just an information-theoretic compression limit. This distinguishes two separate bottlenecks: the information capacity of N vision tokens, and the physical legibility of text rendered at W×H pixels. At 64 tokens (512×512), text above ~900 characters becomes physically blurred; at 100 tokens (640×640), the blur threshold is higher (~1100+ characters). This insight is practical: future compression systems could use higher rendering resolution (maintaining legibility) with more aggressive token compression (reducing token count), potentially pushing the viable compression ratio higher.

Diagnostic value for the field. These measurements provide an empirical reference point that the broader VLM community can use to calibrate their own encoder designs. If a new encoder uses 4,000 tokens to achieve 95% accuracy on documents with 1,000 words, Table 2 tells us that's a 40× compression ratio — far beyond the viable range characterized here — and suggests the encoder is using tokens very inefficiently. Conversely, if a new encoder achieves 95% accuracy with 200 tokens on the same documents, that represents meaningful progress toward the theoretical lower bound.

Connection to the memory-forgetting framework. The compression ratio measurements also directly inform the paper's speculative memory-forgetting proposal (Figure 13). If ~10× compression is near-lossless and ~20× compression retains 60%, then a multi-turn dialogue system could render recent conversation history at 10× compression (high fidelity), medium-term history at 15× compression (partial fidelity), and distant history at 20+× compression (gist only), with the compression ratio smoothly mapping to memory fidelity. The quantitative bounds from Table 2 give this proposal empirical grounding rather than leaving it as pure speculation.

Innovation 4: OCR as a Testbed for Learned Compression with Natural Evaluation Metrics

The paper implicitly demonstrates that OCR tasks provide a uniquely suitable testbed for studying learned compression in VLMs — a methodological insight that, while not explicitly claimed as a contribution, represents a significant practical advance for future research in this direction.

What makes OCR distinctive for compression research. General image compression (e.g., learned image codecs, neural compression) suffers from a fundamental evaluation challenge: the "right" metric is unclear. PSNR and SSIM measure pixel-level fidelity but correlate poorly with human perception; perceptual metrics (LPIPS, FID) capture some aspects of visual quality but are not task-grounded. OCR offers a clean alternative: the compression objective is to preserve text content, and the evaluation metric is text accuracy (precision on Fox benchmarks, edit distance on OmniDocBench). This is a well-defined, objective, and practically meaningful metric that directly measures whether compression preserved the information that matters.

More importantly, OCR provides a natural compression-decompression mapping: the input is an image containing text, and the output is the reconstructed text. The compression ratio is naturally defined as the number of text tokens in the ground truth divided by the number of vision tokens used to encode the image. This creates a clean experimental framework that general image compression lacks — there is no obvious "ground truth information content" metric for a photograph of a cat.

Why this matters as an innovation. The paper doesn't claim this as a contribution — it is framed as "OCR tasks, as an intermediate modality bridging vision and language, provide an ideal testbed for this vision-text compression paradigm" (Section 1) — but the practical implications for the research community are substantial. Anyone who wants to study learned compression in VLMs can follow the paper's methodology: (1) train an encoder-decoder model on document images with text ground truth, (2) vary the vision token count to control the compression ratio, (3) measure text reconstruction accuracy as a function of compression ratio, (4) characterize the tradeoff curve. The benchmarks exist (Fox, OmniDocBench), the metrics are standardized (edit distance, precision), and the paper provides a reference implementation.

This matters because it lowers the barrier to entry for compression research in VLMs. Prior to this work, studying vision token efficiency required building a full VLM and evaluating on general VQA benchmarks, where the relationship between token count and task performance is confounded by the model's reasoning capabilities, not just its compression efficiency. OCR decouples these: if the model can correctly output the text, the compression worked, regardless of whether the model can answer complex questions about the content. This makes OCR a purer probe of compression quality than general VLM benchmarks.

Evidence from OmniDocBench. The OmniDocBench results (Table 3) demonstrate the value of this testbed. The paper can make precise, quantitative comparisons: with 100 vision tokens, DeepSeek-OCR achieves 0.205 overall edit distance, surpassing GOT-OCR2.0 (0.280) which uses 2.56× more tokens. With 256 tokens, it achieves 0.156, outperforming many models using 15–25× more tokens. These numbers are interpretable because the metric (edit distance) directly measures text reconstruction quality — there is no ambiguity about what "better" means. The per-category breakdown in Table 4 further refines this: slides achieve 0.116 edit distance with just 64 tokens, while newspapers require 1,853 tokens (Gundam-M) to achieve 0.077. This tells us that the compression limit is task-dependent — dense multi-column layouts require more tokens than simple single-column text — a finding that would be obscured on general VLM benchmarks.

Limitation of this innovation. The testbed is currently limited to documents with extractable text ground truth. Extending this compression-analysis framework to general visual understanding — where the "ground truth information content" is not easily quantified — remains an open challenge. However, for the specific problem of document processing, the paper has established a replicable methodology that the field can adopt immediately.

Innovation 5: The Demonstration That Aggressive Vision Token Compression Is Viable for Production-Scale Data Generation

The paper's deployment results — 33 million pages per day on 20 nodes (160 A100-40G GPUs), or approximately 200k+ pages per day on a single A100-40G — represent a practical demonstration that low-token-count vision encoding translates directly to throughput gains at production scale, not just theoretical efficiency improvements.

What was known before. The relationship between vision token count and inference throughput is theoretically straightforward — fewer tokens means faster prefill (fewer key-value pairs to compute) and faster generation (fewer tokens for the decoder to attend to). But prior work had not demonstrated this at the scale of millions of documents per day. Pipeline OCR systems (detection → recognition → assembly) can be fast but are fragile and require maintaining multiple expert models. End-to-end VLM-based OCR systems (GOT-OCR2.0, OLMOCR) were slower due to higher token counts. The paper closes this gap by showing that aggressive token compression makes end-to-end VLM OCR fast enough to replace pipeline systems for large-scale data generation.

What makes this demonstration significant. The 33 million pages/day figure is not just a bragging right — it establishes that DeepSeek-OCR can serve as an infrastructure component for LLM/VLM training pipelines. Pretraining datasets for large language models increasingly rely on document parsing to convert PDFs and web documents into training text. If a single institution needs to process 100 million PDF pages for a training corpus, DeepSeek-OCR can complete this in ~3 days on 20 nodes. With a traditional encoder producing ~4,000 tokens per page, the same task would take 5–20× longer (or require proportionally more hardware), potentially making the cost prohibitive.

The paper's choice of an MoE decoder with only 570M activated parameters is directly motivated by this deployment consideration: "it obtains the expressive capability of a 3B model while enjoying the inference efficiency of a 500M small model." This is not an architectural novelty (DeepSeekMoE was introduced in prior work) but a purposeful selection for throughput, demonstrating that the combination of a low-token-count encoder and a low-activated-parameter decoder creates a system that is both accurate enough for production use and fast enough for internet-scale data processing.

Evidence. The throughput claim (200k+ pages/day on a single A100-40G) appears in the abstract and Section 1, though the paper does not provide a detailed throughput analysis (e.g., tokens/second, latency percentiles, batching strategy). The OmniDocBench results (Table 3) provide the accuracy evidence that this throughput doesn't come at the cost of quality: at Base mode (256 tokens), DeepSeek-OCR's 0.156 overall edit distance is competitive with production systems like MinerU2.0 (0.122) and dots.ocr (0.125), while using 20–25× fewer tokens.

Implication for the field. This innovation makes the case that token efficiency is a production metric, not just a research curiosity. The paper's compression ratio experiments (Table 2) establish the theoretical bounds; the deployment throughput demonstrates the practical payoff. Together, they make a compelling argument that the VLM community should treat vision token count as a primary design objective, alongside accuracy, when developing document processing systems — a shift that has not yet happened in the broader field, where token count is typically reported as an incidental statistic rather than an optimization target.

5. Experimental Analysis

Evaluation Methodology

Dataset. The paper uses two primary benchmarks:
- Fox benchmark: An English document dataset (100 pages selected with 600–1300 ground-truth text tokens after tokenization) used for the core compression ratio study (Table 2, Section 4.1). The authors explicitly tokenize ground-truth text using DeepSeek-OCR's own tokenizer (vocabulary size ~129k) to ensure the compression ratio calculation reflects the actual token economy the model experiences.
- OmniDocBench: A diverse PDF document parsing benchmark with comprehensive annotations across multiple document categories (books, slides, financial reports, textbooks, exam papers, magazines, academic papers, notes, newspapers). Used for practical OCR performance evaluation (Tables 3 and 4, Section 4.2). The benchmark provides per-category and overall edit distance metrics.
Base model(s). The decoder is DeepSeek3B-MoE-A570M — a 3-billion-parameter Mixture-of-Experts language model with 570M activated parameters (6 out of 64 routed experts + 2 shared experts activated per token). The encoder is DeepEncoder (~380M parameters: 80M SAM-base + 300M CLIP-large + small convolutional compressor). The choice is motivated by the efficiency-capacity tradeoff: "it obtains the expressive capability of a 3B model while enjoying the inference efficiency of a 500M small model" (Section 3.3). This is critical because the paper's deployment claims (200k+ pages/day on a single A100-40G) depend on low per-token inference cost.
Metrics.
- Precision (%) on Fox benchmark (Table 2): fraction of correctly decoded characters/words, though the paper acknowledges the output format "still cannot completely match Fox benchmarks, so the actual performance would be somewhat higher than the test results" (Section 4.1). This is a limitation — the metric conflates genuine OCR errors with formatting mismatches between model output and ground truth.
- Edit distance on OmniDocBench (Tables 3 and 4): lower values indicate better text reconstruction quality. Edit distance is reported for the overall document and separately for text, formula, table, and reading order components. This decomposition matters because a model might excel at body text recognition while struggling with formula parsing or table structure extraction — the per-component breakdown reveals where compression degrades performance.
- Compression ratio (Table 2, Figure 1a): defined as number of text tokens in ground truth divided by number of vision tokens used. This is the paper's central diagnostic metric. For instance, a document with 800 text tokens decoded from 100 vision tokens has a compression ratio of 8×.
- Vision token count (Table 3, "Tokens" column): average number of vision tokens per page, with valid (non-padding) token counts reported in parentheses where relevant (e.g., "256(182)" means 256 total tokens of which 182 are valid on average).
Baselines. The paper compares against two categories of baselines (Table 3):
- Pipeline models (traditional detection→recognition→assembly): Dolphin, Marker, Mathpix, MinerU-2.1.1, MonkeyOCR-1.2B, PPstructure-v3. These are not end-to-end neural models but multi-stage systems that combine separate layout detection and text recognition components. They are included to show where end-to-end approaches stand relative to production systems.
- End-to-end models: Nougat (2,352 tokens/page), SmolDocling (392 tokens), InternVL2-76B (6,790 tokens), Qwen2.5-VL-7B (3,949 tokens), OLMOCR (3,949 tokens), GOT-OCR2.0 (256 tokens), OCRFlux-3B (3,949 tokens), GPT-4o, InternVL3-78B (6,790 tokens), Qwen2.5-VL-72B (3,949 tokens), dots.ocr (3,949 tokens), Gemini2.5-Pro, MinerU2.0 (6,790 tokens), dots.ocr at 200dpi (5,545 tokens). The critical comparison is DeepSeek-OCR at various token counts (64, 100, 256, 400, 795, 1,853) against these baselines which use widely varying token budgets. The paper's argument rests on achieving competitive accuracy with far fewer tokens.
Generation budget / compute accounting. The "budget" is measured as the number of vision tokens output by DeepEncoder for a given resolution mode — not in FLOPs or wall-clock time. Modes are: Tiny (64 tokens), Small (100 tokens), Base (256 tokens, ~182 valid on average), Large (400 tokens, ~285 valid), Gundam (n×100+256 tokens, typically 795 on OmniDocBench), Gundam-M at 200dpi (n×256+400 tokens, typically 1,853). This accounting is clean for compression experiments but does not account for the cost of the SAM module's window-attention processing (which processes 4,096 patches for a 1024×1024 image before compression) — the "cost" metric captures only the decoder-facing token count, not the encoder's internal computation.
Cross-validation / statistical protocol. None reported. The Fox benchmark experiments (Table 2) use 100 selected pages with 600–1300 text tokens, but there is no mention of cross-validation, multiple runs, or confidence intervals. The OmniDocBench results (Table 3) appear to be single-run evaluations on the standard benchmark split. The paper does not report variance or statistical significance for any result. This is a notable weakness — with only 4–28 pages per text-token bucket in Table 2 (e.g., only 4 pages in the 1200–1300 token range), the precision estimates could have high variance that is not characterized.

Main Quantitative Results

Compression Ratio Study on Fox Benchmark

Headline findings. Table 2 and Figure 1(a) characterize the fundamental compression-accuracy tradeoff curve. With 100 vision tokens (Small mode, 640×640 input), DeepSeek-OCR achieves 98.5% precision at 6.7× compression (600–700 text tokens) and maintains 96.8% precision at both 8.5× and 9.7× compression (800–1000 text tokens). Performance degrades at higher compression ratios: 91.5% at 10.6× (1000–1100 text tokens), 89.8% at 11.3× (1100–1200), and drops to 87.1% at 12.6× (1200–1300). With only 64 vision tokens (Tiny mode, 512×512 input), precision is consistently lower: 96.5% at 10.5×, 93.8% at 11.8×, 83.8% at 13.2×, 85.9% at 15.1×, 79.3% at 16.5×, 76.4% at 17.7×, and 59.1% at 19.7×.

The key pattern. Two separate regimes emerge clearly from Table 2:

<10× compression, 100 vision tokens: precision ≥ 91.5% across all text token ranges up to 1,100 tokens. The model is operating in a near-lossless regime where the compressed visual representation preserves sufficient information for accurate decoding. The drop from 98.5% to 91.5% between 700 and 1100 text tokens is gradual, suggesting graceful degradation rather than a sharp threshold.
>10× compression or reduced token count: precision falls more sharply. At 64 tokens with >13× compression, precision drops below 85%; at 19.7× compression, it reaches 59.1%. The paper attributes this partly to physical blurring at low rendering resolution ("long texts become blurred at 512×512 or 640×640 resolution") rather than purely information-theoretic limits. This is an important distinction: the 60% at 20× compression may reflect the input image's pixel-level illegibility, not the model's inability to decode compressed representations — higher rendering resolution with the same 64 vision tokens (via a better compression scheme) might recover higher accuracy.

Sample size caveat. The number of pages per bucket varies dramatically: 7 pages (600–700), 28 pages (700–800, 800–900), 14 pages (900–1000), 11 pages (1000–1100), 8 pages (1100–1200), and only 4 pages (1200–1300). The precision estimates in the 1200–1300 token bucket (59.1% at 64 tokens, 87.1% at 100 tokens) are based on just 4 documents and should be interpreted as indicative rather than statistically reliable. More documents in the high-compression regime would be needed to characterize the tail of the compression-accuracy curve with confidence.

Comparison between 64 and 100 token modes. The gap between 64 and 100 vision tokens widens as text token count increases: at 600–700 text tokens, the gap is only 2 percentage points (96.5% vs. 98.5%); at 1200–1300 text tokens, the gap balloons to 28 percentage points (59.1% vs. 87.1%). This suggests that 100 vision tokens provide enough capacity for documents up to ~1,100 text tokens (where precision is still 91.5%), while 64 tokens saturate much earlier. The effective capacity of the compressed representation is not simply linear in token count — 100 tokens provides more than 100/64 ≈ 1.56× the capacity of 64 tokens at high compression ratios.

What the precision metric misses. The paper states that "the output format still cannot completely match Fox benchmarks, so the actual performance would be somewhat higher than the test results" (Section 4.1). This means reported precision numbers are lower bounds — formatting differences (spacing, line breaks, punctuation handling) that are not genuine OCR errors still count against the model. The true OCR accuracy is likely higher, but the magnitude of this effect is not quantified. This is a recurring issue in OCR evaluation and is not unique to this paper, but it means the precision numbers in Table 2 should not be compared directly to precision numbers from other systems that use different matching criteria.

Practical OCR Performance on OmniDocBench

Headline findings (Table 3). DeepSeek-OCR achieves competitive document parsing performance while using dramatically fewer vision tokens than competitors. The key comparisons:

Small mode (100 tokens, 640×640 resolution): overall edit distance 0.205. Surpasses GOT-OCR2.0 (0.280, using 256 tokens) and is competitive with much larger models like InternVL3-78B (0.161, using 6,790 tokens) and Qwen2.5-VL-72B (0.168, using 3,949 tokens). This is the most dramatic efficiency demonstration: DeepSeek-OCR matches or beats models using 25–68× more vision tokens.
Base mode (256 tokens, ~182 valid, 1024×1024 resolution): overall edit distance 0.156. Outperforms GOT-OCR2.0 (0.280), OLMOCR (0.277), SmolDocling (0.522), Qwen2.5-VL-7B (0.226), and OCRFlux-3B (0.263). Approaches the performance of dots.ocr (0.182), which uses 3,949 tokens (15.4× more).
Large mode (400 tokens, ~285 valid, 1280×1280 resolution): overall edit distance 0.117. Competitive with MinerU2.0 (0.122, using 6,790 tokens), dots.ocr at 200dpi (0.067, using 5,545 tokens), and Gemini2.5-Pro (0.121). Notably, Large mode's edit distance is almost identical to Base mode (0.117 vs. 0.156), suggesting diminishing returns from the additional 144 valid tokens.
Gundam mode (795 tokens, tiled 640 + global 1024): overall edit distance 0.083. Outperforms all pipeline models and all end-to-end models except dots.ocr at 200dpi (0.067) and Gemini2.5-Pro (0.121). Uses 8.5× fewer tokens than MinerU2.0 (6,790) while achieving better performance.
Gundam-M mode at 200dpi (1,853 tokens): overall edit distance 0.077. The best-performing DeepSeek-OCR configuration, surpassing all end-to-end models and nearly matching dots.ocr at 200dpi (0.067) while using 3× fewer tokens.

Per-component analysis (Table 3 breakdown). The overall edit distance conceals significant variation across document components:

Text recognition: DeepSeek-OCR excels here. Large mode achieves 0.054 text edit distance, matching Gemini2.5-Pro (0.055) and trailing only dots.ocr at 200dpi (0.032) and MinerU2.0 (0.045). This is the strongest evidence that compressed vision tokens preserve text content well — the core compression-decompression task works.
Formula parsing: This is a relative weakness. Base mode achieves 0.267 formula edit distance vs. 0.045 for MinerU2.0, 0.032 for dots.ocr at 200dpi, and 0.055 for Gemini2.5-Pro. Even Gundam-M (0.242) lags substantially behind the best pipeline and end-to-end models. This suggests that mathematical notation, with its 2D spatial structure (fractions, exponents, integrals), requires more visual fidelity than aggressive compression preserves — the compressor may be "blurring" fine structural details that are essential for formula recognition.
Table parsing: DeepSeek-OCR performs competitively. Base mode achieves 0.163 table edit distance, outperforming InternVL2-76B (0.547), Qwen2.5-VL-7B (0.598), and GOT-OCR2.0 (0.459). Gundam-M reaches 0.08, competitive with Gemini2.5-Pro (0.119) and dots.ocr at 200dpi (0.092). Table structure (rows, columns, spanning cells) is apparently decodable from compressed tokens, unlike formulas — perhaps because table structure is more regular and predictable.
Reading order: DeepSeek-OCR performs well here. Large mode achieves 0.067 reading order edit distance (though the paper lists "overall text formula table order" columns, the "order" column for Large mode appears as 0.067, which is the "overall" value in the table structure). It is worth noting that for Large mode, the paper reports 0.138 overall, 0.054 text, 0.277 formula, 0.152 table, and 0.067 for "order" — the last value seems to be reading order performance equivalent to the overall metric for other mode rows. Check Table 3: for Large mode, "overall" is 0.138, "text" is 0.054, "formula" is 0.277, "table" is 0.152, and the final column (labeled "order" in the header) is 0.067. This implies reading order is well-preserved even with aggressive compression.

The compression-performance paradox. A striking pattern in Table 3 is the flattening of returns from additional vision tokens. From Tiny (64 tokens) to Small (100 tokens): overall edit distance improves from 0.386 to 0.205 — a massive 0.181 reduction from just 36 additional tokens. From Small (100) to Base (256): improvement drops to 0.049 (0.205 → 0.156), despite adding 156 tokens. From Base (256) to Large (400): improvement is only 0.018 (0.156 → 0.138) for 144 additional tokens. From Large (400) to Gundam (795): improvement is 0.011 (0.138 → 0.127) for 395 additional tokens. The marginal benefit per token drops sharply — the first 100 tokens provide the vast majority of the model's OCR capability, while additional tokens yield progressively smaller gains. This is exactly what one would expect if the compression is near the information-theoretic limit: once the representation captures the essential text content, more tokens add redundant information that doesn't improve decoding.

Per-Document-Category Analysis

Table 4 results. This breakdown by document type (books, slides, financial reports, textbooks, exam papers, magazines, academic papers, notes, newspapers) reveals which document categories benefit most from additional vision tokens:

Slides: Achieve 0.116 edit distance with just 64 tokens (Tiny mode). Performance is essentially saturated — Gundam mode (0.085) is only marginally better. Slides are typically sparse in text with large fonts, making them trivially decodable from minimal vision tokens.
Books, reports, financial reports: Achieve good performance with 100 tokens (Small mode): 0.085, 0.147, and 0.079 edit distance respectively. Base mode (256 tokens) brings these down to 0.037, 0.1, and 0.027 — dramatic improvements for books and financial reports, modest for reports. This suggests that books and financial documents contain denser or more structured content that benefits from additional token capacity.
Academic papers: Require moderate token counts. Small mode achieves 0.187; Base mode reaches 0.176 — a small improvement. Gundam mode (0.153) provides noticeable gains, suggesting that academic paper layouts (multi-column, references, equations) benefit from the increased resolution of tiled processing.
Newspapers: Show the most dramatic dependence on token count. Tiny mode: 0.94 (essentially unusable). Small mode: 0.744. Base mode: 0.645. Large mode: 0.353. Gundam mode: 0.122. Gundam-M: 0.077. The improvement from 0.94 (64 tokens) to 0.077 (1,853 tokens) spans nearly the full range of the edit distance scale. This aligns with the paper's claim that "the text tokens in newspapers are 4–5,000, far exceeding the 10× compression of other modes" (Section 4.2). Dense multi-column layouts with high text density simply exceed the information capacity of 64–400 compressed tokens.
Exam papers: Also show strong token dependence, improving from 0.294 (Tiny) to 0.079 (Gundam-M). Similar to newspapers, exam papers combine dense text, formulas, and structured layouts.
Notes and magazines: Intermediate dependence. Notes improve from 0.297 (Tiny) to 0.1 (Gundam-M); magazines from 0.201 (Tiny) to 0.079 (Gundam-M).

Implication for the compression boundary claim. The per-category analysis validates and refines the paper's central claim about 10× compression as a practical boundary. Categories where text tokens are typically within 10× of vision tokens (slides, books, reports with ~100–1,000 text tokens) achieve good performance at low token counts. Categories exceeding this ratio (newspapers at 4,000–5,000 text tokens) require proportionally more vision tokens — the Gundam-M mode's 1,853 tokens for ~5,000 text tokens represents roughly 2.7× compression, well within the <10× regime. The paper's 10× "limit" is thus validated across document types: whenever the compression ratio is pushed beyond ~10× (e.g., trying to decode a 5,000-token newspaper from 256 Base mode tokens = 19.5× compression), performance collapses, consistent with the Fox benchmark findings.

Comparison to Baselines

Against GOT-OCR2.0 (the most direct comparison). GOT-OCR2.0 uses 256 vision tokens per page and achieves 0.280 overall edit distance on OmniDocBench. DeepSeek-OCR Small mode (100 tokens) achieves 0.205 — 26.8% lower edit distance with 61% fewer vision tokens. DeepSeek-OCR Base mode (256 tokens) achieves 0.156 — 44.3% lower edit distance with the same token budget. This is a clean apples-to-apples comparison at equal token count: DeepSeek-OCR extracts substantially more text information per vision token. The per-component breakdown reveals where the advantage comes from: DeepSeek-OCR Base achieves 0.054 text edit distance vs. GOT-OCR2.0's 0.189 (71.4% improvement), 0.267 formula vs. 0.360 (25.8% improvement), 0.163 table vs. 0.459 (64.5% improvement), and 0.064 reading order vs. 0.141 (54.6% improvement). The advantage is broad across all components, not driven by a single strength.

Against high-token-count models. The comparison with InternVL2-76B (6,790 tokens) is instructive: DeepSeek-OCR Base mode (256 tokens, 26.5× fewer) achieves 0.156 vs. 0.317 overall edit distance. Against Qwen2.5-VL-72B (3,949 tokens, 15.4× fewer): DeepSeek-OCR Gundam (795 tokens, 5× fewer) achieves 0.083 vs. 0.168. These comparisons support the paper's thesis that token count, not model scale, is the bottleneck for document OCR — DeepSeek-OCR's 570M active parameters with aggressive compression outperform models with 70B+ parameters that use many more vision tokens. However, this comparison is confounded by model architecture and training data differences — InternVL2-76B and Qwen2.5-VL-72B are general VLMs not specialized for OCR, so their higher error rates may reflect generalist training rather than inherent token inefficiency.

Against production pipeline systems. DeepSeek-OCR Gundam mode (0.083 overall) outperforms MinerU-2.1.1 (0.097), PPstructure-v3 (0.077), and MonkeyOCR-1.2B (0.094). Gundam-M (0.077) matches PPstructure-v3. While pipeline systems remain competitive and some (MinerU2.0, dots.ocr at 200dpi) achieve lower edit distances, DeepSeek-OCR's advantage is operational simplicity (single model vs. multi-stage pipeline) and token efficiency. For large-scale data generation where throughput matters, matching pipeline accuracy with a single end-to-end model is a practical win even if absolute accuracy is slightly lower.

Ablation Studies and Robustness Checks

Resolution mode ablation (implicit in Tables 3 and 4). While not presented as a formal ablation, the comparison across Tiny (64 tokens), Small (100), Base (256), Large (400), Gundam (795), and Gundam-M (1,853) modes effectively ablates the effect of vision token count on OCR performance. The key finding: performance improves monotonically with token count but with sharply diminishing returns — the largest jump occurs from 64 to 100 tokens (0.386 → 0.205 overall edit distance), while going from 400 to 1,853 tokens (Gundam-M) yields only 0.138 → 0.077 improvement. This suggests that for most document types, the information content of the text can be encoded in ~100–400 vision tokens, and additional tokens provide marginal layout/structure improvements rather than better text recognition.

Compression ratio ablation across text density (Table 2). By testing documents with 600–700, 700–800, 800–900, 900–1000, 1000–1100, 1100–1200, and 1200–1300 text tokens at both 64 and 100 vision tokens, the paper implicitly ablates the effect of compression ratio while holding vision token count constant. The key finding: for a fixed vision token budget, precision degrades as the compression ratio increases (more text tokens to decode from the same visual representation), but the degradation is gradual rather than catastrophic — there is no sharp "cliff" where decoding suddenly fails. This supports the paper's framing of optical compression as a tunable mechanism rather than a binary success/failure condition.

Document category ablation (Table 4). Testing performance across nine document categories at six resolution modes reveals that the compression limit is category-dependent, not universal. Slides, books, and financial reports achieve good performance at 64–100 tokens; newspapers and exam papers require 795+ tokens. This is a critical finding because it means a one-size-fits-all token allocation is suboptimal — easy documents waste tokens, hard documents starve for them. This implicitly supports the paper's broader vision of adaptive compression where token allocation is difficulty-dependent, though the paper does not develop an adaptive mechanism.

Layout vs. non-layout prompt ablation (not performed). The paper mentions (Section 4.1) that the prompt "<image>\nFree OCR." is used to control output format for the Fox benchmark, and that layout vs. non-layout outputs are controlled by different prompts. However, no ablation compares performance with and without layout information in the prompt, which would reveal whether the compression ratio is affected by the complexity of the output format (plain text vs. text + bounding boxes).

Data mixture ablation (not performed). The paper trains with a 70/20/10 OCR/vision/text data mixture but does not ablate this ratio. Given that DeepSeek-OCR maintains general visual understanding (Figure 12), it would be valuable to know whether the 20% vision data allocation could be reduced without degrading OCR performance, or whether removing it entirely would improve OCR at the cost of general vision capabilities. The paper's claim that general vision data is included "mainly to preserve the general vision interface" is not experimentally validated.

Frozen vs. trainable components ablation (not performed). In Stage 2 training, SAM and the compressor are frozen while CLIP is trainable. No ablation compares this against training all components, freezing CLIP, or training SAM but freezing the compressor. The rationale ("SAM does not need further training," "the compressor's downsampling function is primarily geometric") is plausible but unverified.

Gundam-M continued training vs. joint training ablation (not performed). The Gundam-M mode is obtained through continued training rather than joint training because "training it together would slow down the overall training speed." It is unclear whether the performance of Gundam-M mode (0.077 overall edit distance) would improve further with joint training, or whether separate training introduces any degradation relative to a jointly trained model of equivalent capacity.

Critical Assessment

Does the paper demonstrate that optical compression can substitute for orders-of-magnitude more vision tokens?

The paper's headline claim is that DeepSeek-OCR achieves competitive or superior OCR performance while using dramatically fewer vision tokens than existing approaches. This claim is well-supported by the OmniDocBench results (Table 3) but requires careful qualification about what "orders-of-magnitude" means in this context.

The strongest evidence: DeepSeek-OCR Small mode (100 tokens) outperforms GOT-OCR2.0 (256 tokens) — a 2.56× token reduction with better accuracy. Base mode (256 tokens) outperforms InternVL2-76B (6,790 tokens) — a 26.5× reduction. Large mode (400 tokens, 285 valid) outperforms Qwen2.5-VL-72B (3,949 tokens) — a 13.9× reduction. These are genuinely large factors (up to 26.5×), supporting "order-of-magnitude" in the colloquial sense (factor of 10+) if not the literal mathematical sense (factor of 10 exactly).

However, the comparison is confounded by model scale and specialization. DeepSeek-OCR is a dedicated OCR model trained on 70% OCR data; InternVL2-76B and Qwen2.5-VL-72B are general VLMs that also do captioning, VQA, and grounding. The performance gap may partly reflect specialization, not just token efficiency. A fairer comparison would be against another OCR-specialized model with high token counts (e.g., if GOT-OCR2.0 were scaled to use 6,790 tokens — would it outperform DeepSeek-OCR at 256 tokens?). This experiment is not run, so we cannot cleanly isolate token efficiency from task specialization.

Additionally, the claim of "dramatically fewer tokens" is somewhat self-fulfilling: DeepSeek-OCR's encoder was explicitly designed to produce few tokens. Comparing its token count to encoders that were not designed for this metric is valid for establishing the practical benefit but does not isolate whether the gains come from better architecture, better training data, or the compression objective itself.

Does the 10× compression boundary hold as a general principle?

The paper's central empirical claim is that ~97% OCR precision is achievable at compression ratios under 10×, with graceful degradation beyond that. This claim is supported by the Fox benchmark (Table 2) but the supporting evidence is thin in ways that matter for generalization.

Strengths of the evidence: The precision numbers are internally consistent — the 64-token and 100-token modes show monotonic degradation with compression ratio, and the crossover from "near-lossless" (~97%) to "degraded" (~90%) to "poor" (~60%) roughly aligns with the 10× and 20× thresholds. The OmniDocBench per-category results (Table 4) independently validate this: categories with text token counts within 10× of vision token counts (slides: ~100 text tokens / 64 vision tokens = ~1.6×) achieve good performance at low token counts, while categories exceeding 10× (newspapers: ~4,500 text tokens / 400 vision tokens = 11.3×) require higher-resolution modes.

Weaknesses of the evidence:

Tiny sample sizes at high compression ratios. The 10×, 15×, and 20× compression regimes in Table 2 are represented by as few as 4 pages (1200–1300 tokens) and 8 pages (1100–1200). The precision estimates at these extremes are noisy and may not generalize. A single atypically formatted page could swing the 59.1% figure at 19.7× compression by several percentage points.
The "actual performance would be higher" caveat. The paper acknowledges that formatting mismatches artificially lower reported precision. The magnitude of this effect is not quantified and may vary with document complexity — complex documents likely produce more formatting mismatches, meaning the precision degradation at high compression ratios could be partly an artifact of the evaluation methodology rather than genuine OCR failure. If formatting mismatches account for 5–10 percentage points of the reported error, the true OCR accuracy at 20× compression could be 65–70% rather than 59.1%.
Single benchmark, no cross-dataset validation. The compression ratio analysis uses only the Fox benchmark's English subset. Documents in other languages (especially those with more complex scripts like Arabic, Hindi, or Chinese characters) may have different compression ratio limits because their visual information density differs. Chinese documents, for example, pack more semantic information per character than English, potentially requiring more vision tokens per text token at the same compression ratio. This is not tested.
The 10× boundary is an observation, not a derived limit. There is no theoretical model or cross-validation procedure establishing that 10× is a principled boundary rather than an artifact of the specific model architecture and training data. A larger decoder or different training recipe might shift this boundary to 15× or 20×. The paper acknowledges this ("It is reasonable to conjecture that LLMs...would demonstrate more natural integration of such capabilities") but does not test it.
Blurring vs. information capacity confound. The paper notes that performance degradation at 64 tokens for long documents may be due to physical blurring at 512×512 resolution, not the token count per se. This means the 10× boundary could be extended by using higher rendering resolution with the same token count — a test that is not performed. An experiment rendering the same 1,200-token document at 512×512 (64 tokens), 1024×1024 (256 tokens after compression to 64 tokens via a stronger compressor), and 2048×2048 (64 tokens after even stronger compression) would separate the effects of pixel-level legibility from information capacity. Without this, the 10× boundary is partially an artifact of the chosen rendering resolutions.

Overall assessment: The claim that ~10× compression is viable for OCR is well-supported as an empirical observation on this specific model and dataset. The claim that this represents a general principle or fundamental limit is not sufficiently supported. The paper would be strengthened by multi-dataset testing, larger sample sizes at high compression ratios, and experiments that disentangle physical blurring from information-theoretic compression limits.

Does DeepSeek-OCR achieve state-of-the-art performance among end-to-end models?

The paper claims state-of-the-art performance within end-to-end models on OmniDocBench while using the fewest vision tokens (abstract, Figure 1b). This claim is supported with qualifications.

Looking at Table 3's end-to-end model category: DeepSeek-OCR Gundam-M (0.077 overall edit distance) is the best end-to-end model, outperforming dots.ocr at 200dpi (0.067? — wait, dots.ocr at 200dpi achieves 0.067 overall, which is lower/better than 0.077). Checking carefully: dots.ocr at 200dpi reports 0.067 overall edit distance vs. DeepSeek-OCR Gundam-M at 0.077. So dots.ocr at 200dpi outperforms DeepSeek-OCR, though it uses 5,545 vision tokens vs. 1,853. Gemini2.5-Pro is listed with 0.121 overall but its token count is marked as "-" (proprietary API, unknown). So the precise claim should be: DeepSeek-OCR achieves the second-best performance among end-to-end models (after dots.ocr at 200dpi) while using the fewest vision tokens (among models where token count is known).

The paper's phrasing "state-of-the-art performance within end-to-end models on OmniDocBench while using the fewest vision tokens" is a conjunctive claim — state-of-the-art and fewest tokens — which is true if we interpret "state-of-the-art" as "among the very best" rather than "the single best." DeepSeek-OCR Gundam-M is within 0.01 edit distance of the best known result (dots.ocr at 200dpi) while using 3× fewer tokens.

A missing comparison: the paper does not report DeepSeek-OCR with the same token budget as the top performers (~5,500 tokens) to see if it would surpass them. If DeepSeek-OCR could scale to 5,500 tokens and achieve 0.05 or lower edit distance, that would strengthen the claim. The Gundam-Master mode at 200dpi (1,853 tokens, 0.077) already suggests the trend is favorable — a hypothetical "Gundam-Master at 300dpi" with ~4,000 tokens might close the remaining gap to dots.ocr.

Does the paper demonstrate that compact language models can effectively learn to decode compressed visual representations?

This claim — that the compression-decompression mapping is learnable by small models — is strongly supported by all the main results. The 570M-active-parameter decoder achieves high-fidelity decoding across diverse document types, languages, and specialized content (charts, chemical formulas, geometry). The claim is a direct demonstration: the model was trained and it works. The question is not whether the claim is true (it is) but whether the paper establishes the lower bound correctly — i.e., how much smaller could the decoder be and still work, and how much better would a larger decoder perform?

The paper speculates that larger LLMs would perform better ("It is reasonable to conjecture that LLMs...would demonstrate more natural integration of such capabilities") but provides no evidence. A scaling experiment — testing decoders of different sizes (e.g., 100M, 570M, 1.5B, 3B active parameters) on the same compression task — would characterize the decoder-capacity dimension of the compression-accuracy curve. Without this, we know the 570M decoder works well at 10× compression but don't know whether a 200M decoder would also work, or whether a 3B-active decoder would push the viable compression ratio to 15× or 20×. This is the most obvious missing experiment for establishing the compression paradigm's scalability.

Missing experiments that would significantly strengthen the paper

Decoder scaling study. Train the same DeepEncoder with decoders of varying activated parameter counts (100M, 300M, 570M, 1.5B, 3B) and measure the compression-accuracy curve for each. This would reveal whether decoder capacity is a bottleneck and establish scaling laws for optical compression analogous to the pretraining scaling laws the field already understands.
Cross-dataset compression boundary validation. Test the compression ratio hypothesis on non-English documents (Chinese, Arabic, Hindi) and on non-document text images (scene text, handwritten text) to establish whether the ~10× boundary is universal or language/domain-specific.
Rendering resolution vs. token count disentanglement. For a fixed document with 1,200+ text tokens, render at multiple resolutions (512×512, 1024×1024, 2048×2048) and compress each to the same number of vision tokens (e.g., 64) by varying the compressor's downsampling factor. This would isolate the effect of pixel-level legibility from information-theoretic compression capacity.
Training data ablation. Systematically vary the data mixture (100% OCR, 90/10, 80/20, 70/20/10) and measure both OCR accuracy and general vision capability retention. This would validate the claimed benefit of the 20% vision data allocation.
Component freezing ablation. Compare Stage 2 training with different freezing configurations (all frozen, SAM frozen, compressor frozen, CLIP frozen, all trainable) to validate the specific choice of freezing SAM+compressor while training CLIP.
Comparison against OCR-specialized high-token variants. Train or scale an existing OCR-specialized model (e.g., GOT-OCR2.0) to use 2,000–4,000 tokens and compare against DeepSeek-OCR at matched accuracy to verify that the performance advantage persists when controlling for task specialization, not just token count.
Throughput analysis with variance estimates. Report tokens/second, latency percentiles (p50, p95, p99), and batch size scaling for the deployment throughput claim. The current claim (200k+ pages/day on a single A100-40G) is not decomposed into measurable components.
Statistical significance on Fox benchmark. Given the tiny sample sizes (4–28 pages per bucket), report confidence intervals or bootstrapped variance estimates for the precision numbers in Table 2.

What the paper does and does not demonstrate

What it demonstrates convincingly:

A specifically designed serial compression architecture (DeepEncoder) can produce dramatically fewer vision tokens than existing VLM encoders while preserving decodable text information. The architectural contribution is real and well-motivated.
A compact MoE decoder (570M active parameters) can learn to accurately decompress text from these compressed vision tokens across diverse document types and languages. The decompression mapping is learnable at practical model scales.
For this specific model and data mixture, there exists a measurable compression-accuracy tradeoff curve with ~97% precision at <10× compression and ~60% precision at ~20× compression on English documents.

What it suggests but does not conclusively demonstrate:

That 10× represents a fundamental compression boundary for optical text encoding, rather than an artifact of the specific architecture, training recipe, and evaluation setup.
That the compression approach generalizes to non-English scripts, non-document text, or documents with extreme text density beyond the tested range.
That larger decoders would substantially improve the compression-accuracy tradeoff (the conjecture is plausible but untested).
That optical compression as a general memory mechanism for LLMs (Figure 13) would work in practice beyond the OCR proof-of-concept — this requires experiments on multi-turn dialogue, needle-in-a-haystack tasks, and long-document question answering that are not performed here.

What it does not address:

The latency cost of encoding. The token count comparison against baselines assumes token count is the dominant cost, but DeepEncoder's two-stage processing (SAM window attention + CLIP global attention) has its own latency characteristics that are not compared to, say, a single ViT forward pass at lower resolution. If DeepEncoder takes 3× longer to encode than a ViT but produces 10× fewer tokens, the total inference time advantage may be less than the token count ratio suggests.
The difficulty estimation problem. The paper's broader vision (adaptive compression based on document complexity) requires knowing how many tokens a document needs before encoding it — a chicken-and-egg problem not addressed here.
Any mechanism for combining the compression encoder with standard LLM training (e.g., interleaved optical-text pretraining). The paper proposes this as future work but provides no results.

6. Limitations and Trade-offs

6.1 The Compression Ratio Boundary Is Characterized on a Single Benchmark with Tiny Sample Sizes at High Compression

The assumption or constraint. The paper's central empirical claim — that optical text decoding achieves ~97% precision at <10× compression and ~60% at ~20× compression — is derived entirely from the Fox benchmark's English subset using 100 pages. The sample sizes at high compression ratios are extremely small: only 4 pages in the 1200–1300 text token bucket, 8 pages in the 1100–1200 bucket, and 11 pages in the 1000–1100 bucket (Table 2). The paper acknowledges a measurement artifact that further weakens these estimates:

"the output format still cannot completely match Fox benchmarks, so the actual performance would be somewhat higher than the test results" (Section 4.1)

This means the reported precision numbers conflate genuine OCR errors with formatting mismatches (spacing, line breaks, punctuation conventions), but the magnitude of this effect is not quantified and may vary systematically with document complexity — longer, more complex documents likely produce more formatting mismatches.

The consequence. The headline numbers that define the entire research program — 97% at 10× compression, 60% at 20× — are noisy point estimates with unknown variance. A shift of ±5 percentage points at 20× compression would substantially change the viability assessment for memory-forgetting applications. More critically, we cannot distinguish whether the degradation from 97% to 60% represents a genuine information-theoretic limit of optical compression or an artifact of (a) the specific model architecture, (b) the rendering resolution choices (512×512 and 640×640), (c) the evaluation methodology, or (d) random sampling noise from 4-page buckets. A practitioner trying to decide whether to invest in optical compression for a production system cannot assess whether the 10× boundary will hold on their document distribution.

What evidence exists in the paper. Table 2 provides the raw precision numbers and sample sizes. The paper does not report confidence intervals, bootstrapped variance estimates, or cross-validation results on the Fox benchmark. The OmniDocBench per-category analysis (Table 4) provides an independent qualitative validation that compression limits are category-dependent (newspapers fail at low token counts while slides succeed), but this is categorical (document type) rather than quantitative (compression ratio), so it corroborates the general phenomenon without validating the specific 10× threshold. There are no experiments on non-English documents, scene text, handwritten text, or other text domains that would test whether the compression boundary generalizes beyond English document PDFs rendered at specific resolutions.

Mitigation status. The paper does not address this limitation beyond acknowledging the formatting mismatch issue. There is no suggestion of cross-validation, multi-dataset replication, or statistical characterization of the compression-accuracy tradeoff curve. The authors frame the work as "preliminary validation" (Section 6) and explicitly call for "needle-in-a-haystack testing, and other evaluations in the future," which implicitly acknowledges the narrowness of the current evaluation, but this doesn't mitigate the weakness of the existing evidence for the paper's primary empirical claim.

6.2 Difficulty Estimation Cost Is Unaccounted for in Any Practical Deployment Scenario

The assumption or constraint. The paper's broader vision (Section 5, Figure 13) envisions adaptive compression where different documents or different temporal stages of conversation history receive different vision token budgets — recent context at high resolution, distant context at low resolution. However, determining the appropriate compression level for a given document requires knowing approximately how many text tokens it contains before encoding it, which is a chicken-and-egg problem: you need to decode the text to know how dense it is, but you want to set the compression ratio to avoid wasting tokens on sparse documents or starving dense documents of tokens.

The paper's multi-resolution design (Section 3.2.2) sidesteps this by having the user or system administrator manually select the resolution mode (Tiny/Small/Base/Large/Gundam), but this is not adaptive — the same mode is applied to all documents regardless of their information density. The per-category analysis (Table 4) reveals the cost of non-adaptive allocation starkly: Tiny mode (64 tokens) achieves 0.116 edit distance on slides but 0.94 on newspapers — a factor of 8.1× worse performance. A system that uniformly applies Tiny mode to all documents would catastrophically fail on newspapers while wasting capacity on slides (which don't need even 64 tokens).

The consequence. Any practical deployment of the adaptive compression vision requires a difficulty estimation mechanism that the paper does not provide. The cost of this estimation — whether it involves a pre-processing pass, a lightweight classifier, or progressive encoding at multiple resolutions — would need to be amortized into the total compute budget. For the paper's headline throughput claim (200k+ pages/day on a single A100-40G), this overhead is not accounted for. If difficulty estimation requires generating even a small number of extra vision tokens (e.g., encoding a low-res preview to estimate text density), the effective throughput would be lower. If estimation is inaccurate, the system misallocates tokens and loses accuracy on dense documents. The paper provides no analysis of what an acceptable difficulty estimator would cost or how accurate it would need to be to preserve the reported compression-accuracy tradeoff.

What evidence exists in the paper. Table 4 provides the clearest evidence for the necessity of adaptive allocation: performance varies dramatically across document categories at a given token budget. The paper explicitly notes this pattern: "some categories of documents require very few tokens to achieve satisfactory performance, such as slides which only need 64 vision tokens... For newspapers, Gundam or even Gundam-master mode is required to achieve acceptable edit distances" (Section 4.2). The paper also notes that the 10× compression boundary explains this variation: "the text tokens in newspapers are 4–5,000, far exceeding the 10× compression of other modes," but does not propose a method for estimating this text token count before encoding.

Mitigation status. Not addressed. The paper treats resolution mode selection as a manual configuration choice, not an adaptive decision. The memory-forgetting framework (Figure 13) implicitly assumes that temporal distance, not information density, determines compression level — a reasonable heuristic for conversational memory (older messages are compressed more) but not for document processing where content density varies independently of temporal factors. The paper does not discuss difficulty estimation as a research problem, propose any lightweight estimation method, or analyze the sensitivity of overall accuracy to estimation errors.

The assumption or constraint. The paper treats optical compression as a single-pass encode-decode operation: an image is encoded to N vision tokens, and the decoder produces the best text it can from those tokens. If the decoding is inaccurate because the compression ratio was too aggressive (e.g., decoding a newspaper at 400 tokens produces 0.353 edit distance), there is no mechanism to recover — no way to say "this result is low-confidence, let me re-encode at higher resolution" or "let me focus additional compute on the regions where the decoder was uncertain."

This stands in contrast to how other compression systems handle rate-distortion tradeoffs. Modern video codecs use variable bitrate encoding. Neural compression systems can allocate more bits to high-complexity regions. LLMs with test-time compute scaling can spend more inference budget on harder problems. DeepSeek-OCR has none of these mechanisms: the token budget is fixed before encoding, and failure is silent — the decoder produces output regardless of whether the compressed representation contained sufficient information.

The consequence. In any deployment where document complexity varies, the system faces an unresolvable tension. To guarantee acceptable accuracy on the hardest documents (newspapers, dense academic papers), the token budget must be set high enough for those documents — wasting tokens on the majority of simpler documents. To minimize average token consumption, the budget must be set for typical documents — producing unacceptable errors on the hardest cases. Without a refinement mechanism, the system cannot adapt after seeing the initial decoding result, even though the decoder's output could in principle signal its own uncertainty (e.g., through low-confidence predictions, garbled output, or explicit uncertainty tokens — none of which the model is trained to produce).

What evidence exists in the paper. The flattening edit distance curve across resolution modes (Table 3: 0.386 → 0.205 → 0.156 → 0.138 → 0.127 → 0.077 from Tiny to Gundam-M) shows that additional tokens provide diminishing returns, which means over-allocating tokens to simple documents is wasteful. The per-category breakdown (Table 4) shows that the optimal token budget varies by a factor of ~28× between slides (64 tokens sufficient) and newspapers (1,853 tokens required for good performance). The absence of any adaptive or iterative mechanism means the system cannot exploit this variance — it must choose a single operating point.

Mitigation status. Not addressed. The paper's architecture has no feedback loop from decoder to encoder, no confidence estimation mechanism, and no iterative refinement capability. The memory-forgetting framework (Figure 13) envisions multi-level compression where older text is stored at progressively lower resolutions, but this is a one-way degradation (text is compressed once and stored), not an adaptive refinement loop. The paper's suggestion for future work (Section 5) focuses on "digital-optical text interleaved pretraining" and "needle-in-a-haystack testing" — evaluating the compression paradigm, not building adaptive mechanisms within it.

6.4 Formula Parsing Remains Fundamentally Degraded Under Aggressive Compression

The assumption or constraint. The paper's compression approach assumes that text information can survive aggressive spatial downsampling because text is highly redundant — neighboring pixels within a character convey similar information, and the language model can reconstruct characters from partial visual evidence. This assumption breaks down for content where fine 2D spatial structure is semantically meaningful: mathematical formulas, chemical structures, and geometric diagrams rely on precise relative positioning of small symbols (exponents, subscripts, fraction bars, integral signs, bond angles) that may be lost or distorted by the 16× convolutional compressor.

The paper trains on chemical formulas and geometric figures (Section 3.4.2), so the model is exposed to this content during training, but the architecture's compression bottleneck may be fundamentally lossy for these modalities in ways that no amount of training can recover.

The consequence. The OmniDocBench formula parsing results reveal this limitation clearly: DeepSeek-OCR's formula edit distance (0.267 at Base mode, 0.277 at Large mode, 0.269 at Gundam, 0.242 at Gundam-M) is systematically worse than its text edit distance (0.054 at Large mode) and lags substantially behind the best pipeline systems (MinerU2.0: 0.045, dots.ocr at 200dpi: 0.032). Even at the highest resolution mode (Gundam-M, 1,853 tokens), formula edit distance (0.242) is ~5× worse than the best pipeline systems. This is not simply a token budget problem — Gundam-M uses more tokens than most other end-to-end models, yet formula accuracy remains poor relative to pipeline approaches. The compression architecture appears to be fundamentally lossy for the fine spatial structure that formulas require, and adding more tokens does not recover the lost information because the damage occurs at the compression stage before the token count is determined.

What evidence exists in the paper. Table 3 provides the formula edit distance across all DeepSeek-OCR modes and all baselines. The per-category analysis (Table 4) does not separately report formula performance, so we cannot assess whether formula parsing is uniformly bad across document types or specifically problematic in dense academic contexts. The paper does not provide qualitative examples of formula parsing failures, so the specific failure modes (misrecognized symbols, lost structural relationships, incorrect LaTeX/MathML output) are not characterized. The training data includes 5M chemical formula examples (Section 3.4.2), so the weakness is not due to training data scarcity.

Mitigation status. Not addressed directly. The paper does not discuss formula parsing as a specific weakness, propose architectural modifications for formula-sensitive compression (e.g., preserving higher resolution for detected formula regions), or acknowledge that the compression paradigm may be fundamentally limited for content requiring fine 2D spatial reasoning. The "deep parsing" capability (Figures 7, 9, 10) demonstrates that the model can parse formulas when they are the primary content of an image (via the specialized "Parse the figure" prompt), but this uses the same architecture and likely suffers from the same compression-induced degradation — the qualitative examples shown are cherry-picked successes that don't characterize the failure rate.

6.5 The Throughput Claims Are Not Decomposed into Measurable Components, Obscuring Latency-Vs-Token-Count Tradeoffs

The assumption or constraint. The paper makes specific throughput claims: "200k+ pages per day (a single A100-40G)" in the abstract, and "33 million pages of data per day for LLMs or VLMs using 20 nodes (each with 8 A100-40G GPUs)" in Section 1. These numbers are presented as evidence that low vision token counts translate to practical deployment efficiency. However, the paper provides no decomposition of these throughput numbers into measurable components: What is the tokens-per-second throughput? What are the latency distributions (p50, p95, p99)? What batch size achieves this throughput? What is the breakdown of time between the encoder forward pass (DeepEncoder) and the decoder autoregressive generation?

This matters because token count is not the only determinant of inference speed. DeepEncoder's two-stage processing — SAM window attention on 4096 patches followed by CLIP global attention on 256 tokens — has different latency characteristics than a single ViT forward pass producing the same number of output tokens. The SAM module processes 4096 tokens with window attention; while window attention scales linearly rather than quadratically, it is still processing 16× more tokens than the CLIP module, and this computation is on the critical path before any decoder tokens can be generated. The decompression decoding is autoregressive — the decoder generates text tokens one at a time, each attending to all vision tokens — so the generation time scales with both the number of output text tokens and the number of vision tokens. A document with 5,000 text tokens will take 50× longer to decode than a document with 100 text tokens, regardless of how few vision tokens were used.

The consequence. A practitioner evaluating DeepSeek-OCR for production use cannot determine whether the throughput advantage over, say, GOT-OCR2.0 (256 tokens, unknown encoding architecture) comes primarily from faster encoding, faster decoding, or both. More critically, the latency characteristics are completely unknown. If DeepEncoder's two-stage processing adds 200ms of encoding latency regardless of output token count, then for short documents (where decoding is fast), the encoding latency dominates and the token count reduction provides less wall-clock benefit than the token count ratio suggests. For the adaptive compression use case — where some documents receive aggressive compression and fast decoding while others receive mild compression and slower decoding — the variance in per-document processing time is not characterized, making it impossible to provision hardware for latency-sensitive applications.

What evidence exists in the paper. Only the aggregate throughput claims and the training infrastructure description (Section 3.5.2): 20 nodes × 8 A100-40G GPUs, pipeline parallelism = 4, data parallelism = 40, global batch size = 640. The paper reports training speed (90B tokens/day for text-only, 70B tokens/day for multimodal) but not inference speed. The abstract states "200k+ pages per day (a single A100-40G)" without specifying page characteristics (average text tokens, resolution mode, batch size, or whether this includes encoding + decoding or only decoding). The 33 million pages/day claim uses the same 20-node configuration as training but for inference, yet inference and training have different batching and memory characteristics (no backward pass, no optimizer states, potential for much larger batch sizes) that are not discussed.

Mitigation status. Not addressed. The paper does not provide an inference throughput analysis, latency benchmarks, or scaling curves for batch size vs. throughput. The claims are presented as headline numbers without the supporting detail that would allow independent verification or cost estimation for different deployment scenarios. The paper's focus is on the compression research, not the engineering of the inference system, but the throughput numbers are prominently featured in the abstract and introduction as evidence of practical value, making their imprecision a meaningful limitation.

6.6 The Model Is Not a Chatbot and Requires Prompt Engineering, Limiting Integration with Conversational LLM Systems

The assumption or constraint. DeepSeek-OCR is trained only through pretraining-style next-token prediction on OCR, vision, and text data — it does not undergo supervised fine-tuning (SFT) or reinforcement learning from human feedback (RLHF). The paper explicitly states:

"Note that since we do not include SFT (Supervised Fine-Tuning) stage, the model is not a chatbot, and some capabilities need completion prompts to be activated" (Section 4.3.3, Figure 12 caption)

This means DeepSeek-OCR is a base model that requires specific prompt formats to elicit desired behaviors. For example, plain text OCR requires the prompt "<image>\nFree OCR." while layout-aware parsing requires "<image>\n<|grounding|>Convert the document to markdown." The model does not engage in multi-turn dialogue, cannot follow conversational instructions ("can you extract the text from the table on page 3?"), and may produce unpredictable outputs when given prompts that deviate from its training distribution.

The consequence. The paper's broader vision — using optical compression as a memory mechanism for LLMs in multi-turn conversations (Figure 13, Section 5) — implicitly requires the OCR model to integrate with an LLM that manages the conversation. But DeepSeek-OCR in its current form cannot serve as a modular "memory decoder" that an LLM queries conversationally. An LLM could theoretically generate the appropriate prompt to extract text from a compressed visual memory, but this requires the LLM to know the exact prompt format ("<image>\nFree OCR.") and to format its request accordingly — a brittle interface that would break if the prompt format changes or if the memory content requires a different mode (e.g., the memory happens to contain a chart that should be parsed as an HTML table).

More fundamentally, the model's lack of conversational fine-tuning means it cannot handle ambiguous or under-specified requests. If an LLM asks "what was discussed in the conversation from last week?" and the compressed visual memory contains text that the model can decode, DeepSeek-OCR can only output the raw decoded text — it cannot summarize, filter by relevance, or engage in back-and-forth clarification. This means the optical compression pipeline, as demonstrated, is a text extraction tool rather than an integrated memory component, and substantial additional engineering (fine-tuning for instruction following, developing a robust API between the LLM and OCR model, handling mode selection) would be required to realize the memory-forgetting vision.

What evidence exists in the paper. The paper's qualitative examples (Figures 7–12) demonstrate the model's capabilities under carefully constructed prompts, but also reveal the brittleness: different tasks require different prompts ("Free OCR.", "<|grounding|>Convert the document to markdown.", "Parse the figure.", "Describe this image in detail.", "Locate <|ref|>object<|/ref|> in the image."), and the model does not infer the desired output format from conversational context. The paper does not evaluate the model's robustness to prompt variations (e.g., "extract the text" vs. "OCR this document" vs. "read this page") or its ability to handle multi-turn interactions where the user refines the request based on initial output. The open-source release provides model weights but not a chat-tuned variant, confirming that conversational capability is not part of the current offering.

Mitigation status. The paper acknowledges this limitation explicitly in the Figure 12 caption. The inclusion of general vision data (20% of training) and text-only data (10%) is described as preserving "the general vision interface, so that researchers interested in our model and general vision task can conveniently advance their work in the future" (Section 3.4.3), which implies that SFT could be added by downstream users. However, the paper provides no guidance on how to fine-tune the model for conversational use, what data would be needed, or whether fine-tuning would degrade OCR accuracy (a common catastrophic forgetting concern). The limitation is thus acknowledged but not addressed, and the gap between the demonstrated tool (a prompt-controlled OCR model) and the envisioned application (an integrated memory component for conversational agents) remains substantial.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a methodological reframing with architectural consequences — not a paradigm shift in the Kuhnian sense, but a genuine inversion of how the VLM community should think about vision token count. Prior to DeepSeek-OCR, vision token count was treated as an incidental engineering artifact: you chose an encoder architecture, and the token count followed from its design (patch size, resolution, tiling strategy). The optimization objective was accuracy; token count was a cost to be paid. This paper makes a case that token count should be a first-class design variable, and that a carefully architected encoder can push the compression ratio to ~10× near-lossless and ~20× partially-lossless on document text — establishing quantitative reference points where none existed before.

The magnitude of this shift is diagnostic rather than revolutionary. The paper does not claim to have solved long-context processing for LLMs, nor does it demonstrate that optical compression works in the conversational memory setting it speculates about. What it does is provide (1) a clear empirical characterization of the compression-accuracy tradeoff on document OCR, (2) an encoder architecture that achieves state-competitive accuracy with 15–65× fewer vision tokens than widely-used alternatives, and (3) a conceptual framework — the compression ratio as a primary metric — that other researchers can adopt immediately. These contributions change the conversation from "how do we handle too many vision tokens?" (a downstream engineering problem) to "how few vision tokens do we actually need?" (a research question with measurable answers).

Resolving prior contradictions. The paper implicitly reconciles a tension in the VLM literature that the authors do not explicitly name but that runs through their critique of existing encoders (Section 2.1, Figure 2). The tension is between two competing claims: (a) high-resolution images are necessary for document OCR because small text requires fine spatial detail, and (b) high-resolution encoding produces too many vision tokens, degrading inference speed and limiting context length. Prior work accepted this as an unavoidable tradeoff — you can have accurate OCR or efficient processing, but not both. DeepSeek-OCR's serial compression architecture demonstrates that this tradeoff is not fundamental: the SAM module handles the high-resolution perception (capturing fine text detail), while the 16× compressor and CLIP module produce a compact representation that the decoder can efficiently process. The architecture effectively decouples the resolution needed for visual perception from the token count presented to the language model.

Research directions that become more attractive:

Compression-first encoder design. Before this paper, an encoder researcher optimizing for VLM accuracy would increase resolution, add more ViT layers, or scale up parameters — all of which increase token count. After this paper, the natural next question for any new encoder is: "what compression ratio does it achieve, and what is its compression-accuracy curve?" This reframes encoder evaluation from a single accuracy number to a Pareto frontier.
Token-aware data mixing and training. The paper's finding that different document categories require dramatically different token budgets (slides: 64 tokens sufficient; newspapers: 1,853 tokens required; Table 4) suggests that training data mixtures should be optimized not just for task coverage but for token efficiency — including too many "easy" examples may waste training compute on redundant token-rich representations.
Optical compression as a general mechanism for LLM context extension. The paper's speculative memory-forgetting framework (Figure 13) is now empirically grounded: the ~10× near-lossless compression bound provides a concrete multiplier for how much effective context an LLM could gain by storing historical text as compressed visual tokens. A system storing the last 10,000 tokens of conversation as text and the previous 100,000 tokens as compressed images at 10× compression effectively extends the context window by 10× without architectural changes to the LLM.
Learned compression beyond OCR. The methodology — train an encoder-decoder on (image, text) pairs, vary the bottleneck token count, measure reconstruction accuracy — can be applied to other domains where images contain structured information that can be evaluated with objective metrics: code screenshots → code text, sheet music → notation, diagrams → structured descriptions, maps → routing instructions. The paper provides a replicable experimental template.

Research directions that become less attractive:

Indiscriminate resolution scaling for document VLMs. The paper's results make it harder to justify architectures that scale vision tokens linearly with image area without a compression mechanism. A new document VLM that uses 7,000 tokens per page (like InternVL2-76B or MinerU2.0) must now explain why it needs 25–65× more tokens than DeepSeek-OCR to achieve comparable accuracy — or demonstrate that those extra tokens unlock capabilities (e.g., extremely fine-grained spatial reasoning) that aggressive compression loses. The burden of proof has shifted.
Pipeline OCR systems as the default for large-scale data generation. The paper demonstrates that an end-to-end model with aggressive token compression can match or exceed pipeline system accuracy (MinerU, PPstructure) while being simpler to deploy and maintain. For organizations building document processing pipelines, the cost-benefit analysis now tilts toward end-to-end compressed models, especially when throughput matters and the deployment simplicity of a single model outweighs the marginal accuracy gains of multi-stage pipelines.

Follow-Up Research This Work Enables

Decoder scaling laws for optical compression. The paper uses a single decoder (DeepSeek3B-MoE-A570M) and speculates that larger LLMs would achieve better decompression. A direct follow-up would train the same frozen DeepEncoder with decoders of varying activated parameter counts (e.g., 100M, 300M, 570M, 1.5B, 3B, 7B) on identical OCR data, then measure the compression-accuracy curve for each. The key question: does decoder capacity shift the viable compression ratio upward (e.g., 15× near-lossless with a 3B-active decoder vs. 10× with 570M), and does it follow a predictable scaling law? A negative result — finding that decoder capacity above 570M provides no improvement — would indicate that the compression bottleneck is in the encoder, not the decoder, redirecting research toward encoder architecture rather than decoder scale.

Cross-script compression boundary characterization. The paper's compression ratio analysis (Table 2) is English-only, yet the information density of text varies substantially across writing systems. A logographic script like Chinese packs roughly 2–3× more semantic information per character than alphabetic English, but Chinese characters also have more complex visual structure per character. A direct replication of the Fox benchmark methodology on Chinese, Arabic (cursive, right-to-left, character shaping), and Hindi (Devanagari, complex conjuncts) documents would test whether the ~10× compression boundary is universal or script-dependent. The experiment requires: (1) selecting documents with 600–1300 tokens across languages, (2) rendering at 640×640, (3) measuring precision at 64 and 100 vision tokens from the same DeepSeek-OCR model. A finding that Chinese requires 2–3× more vision tokens per text token would refine the compression boundary from a universal constant to a script-specific parameter.

Rendering-resolution-vs-compression disentanglement experiment. The paper notes that performance degradation at high compression ratios may be due to physical blurring of text at low rendering resolution (512×512, 640×640), not the information capacity of the compressed tokens. A clean experiment to separate these effects: take a fixed set of documents with 1,000–1,200 text tokens, render each at multiple resolutions (512×512, 1024×1024, 2048×2048), and compress each rendering to the same number of vision tokens (e.g., 64) by varying the compressor's downsampling factor (16×, 64×, 256×). If compression ratio — not rendering resolution — determines accuracy, performance should be identical across rendering resolutions at the same token count. If rendering resolution matters, there is a legibility floor that no amount of compression cleverness can bypass — this would establish a fundamental limit for the optical compression approach and quantify how much headroom remains from better rendering alone.

Adaptive token allocation with oracle difficulty estimation. The per-category results (Table 4) show that optimal token budget varies by ~28× across document types, but the paper provides no mechanism for selecting the right budget per document. A follow-up would implement an oracle adaptive system: use ground-truth text token count (known from the OmniDocBench annotations) to select the minimum resolution mode that keeps the compression ratio under 10× for each document, then measure overall edit distance and average token consumption. Compare this against the fixed-mode baselines from Table 3. The key metric: can adaptive allocation match Gundam-mode accuracy (0.083 overall) while using substantially fewer average tokens? A positive result — achieving 0.10–0.12 edit distance with ~200 average tokens instead of 795 — would demonstrate that the adaptive compression vision is viable with perfect difficulty estimation, motivating investment in practical difficulty estimators. A negative result — adaptive allocation performs no better than fixed Base mode — would indicate that token count is only loosely correlated with document difficulty and that other factors (layout complexity, image quality, font variation) dominate.

Formula-aware compression architecture. The OmniDocBench results reveal formula parsing as DeepSeek-OCR's clear weakness: formula edit distance (0.242–0.277) is 4–5× worse than text edit distance (0.054) even at the highest resolution mode. A targeted follow-up would modify DeepEncoder to preserve higher spatial resolution specifically for formula regions: add a formula detection head (trained on the 5M chemical formula data from Section 3.4.2) that outputs a spatial attention mask identifying regions likely to contain formulas, then apply less aggressive compression (e.g., 4× instead of 16×) to those regions while maintaining the standard 16× compression elsewhere. The evaluation would compare formula edit distance on OmniDocBench against the unmodified DeepSeek-OCR at matched average token counts. A substantial improvement in formula accuracy without degrading text or table performance would validate that the compression bottleneck, not training data or decoder capacity, is the limiting factor for formula recognition.

Multi-turn memory decoding with needle-in-a-haystack evaluation. The paper's most speculative claim — that optical compression can serve as a memory mechanism for LLMs (Figure 13, Section 5) — is untested. A concrete follow-up experiment: (1) conduct a synthetic multi-turn dialogue with 50 turns (~10,000 tokens), (2) render turns 1–40 as a single document image at 1024×1024 (Base mode, 256 tokens, ~10× compression), (3) keep turns 41–50 as raw text in the context, (4) prepend the compressed image tokens + prompt "Here is a summary of the earlier conversation: <image>\nFree OCR." to the text context, (5) ask factual questions about turn 5, turn 25, and turn 45 (needle-in-a-haystack). Compare against a baseline where turns 1–40 are stored as full text tokens (consuming the full context window). The key metric: can the LLM + DeepSeek-OCR pipeline answer questions about early conversation turns that would be outside the text-only context window, and at what accuracy? A positive result (e.g., 80%+ accuracy on turn 5 questions) would provide the first evidence that optical compression works for conversational memory, not just document OCR. A negative result (near-chance accuracy) would bound the current model's capability and clarify what improvements (larger decoder, better rendering, specialized training) are needed.

Practical Applications and Downstream Use Cases

Large-scale pretraining data generation for LLMs and VLMs. The paper's throughput claim — 200k+ pages per day on a single A100-40G GPU, or 33 million pages per day on 20 nodes — positions DeepSeek-OCR as an immediate drop-in replacement for pipeline OCR systems in training data pipelines. An organization curating a 100-million-page document corpus for LLM pretraining can process it in ~3 days on 20 nodes, with competitive accuracy to production pipeline systems (MinerU2.0: 0.122 overall edit distance; DeepSeek-OCR Gundam-M: 0.077). The practical benefit is not just speed but operational simplicity: a single model replaces the detection→recognition→assembly pipeline, eliminating failure propagation between stages and reducing maintenance burden. The open-source release makes this immediately actionable — any research lab with A100 GPUs can deploy this today.

On-device document scanning with adaptive quality. The multi-resolution capability (64–400 tokens in a single model) enables a practical mobile document scanning application where the user's device adjusts OCR quality based on battery, latency, and document complexity constraints. A quick preview scan at Tiny mode (64 tokens, 512×512) provides instant (~100ms) text extraction for simple documents like receipts or single-column text. If the extracted text appears garbled or truncated (a simple heuristic: character-level perplexity or repeated nonsense tokens), the system automatically re-scans at Base mode (256 tokens). If the user explicitly requests high-quality extraction (e.g., for a dense academic paper), the system uses Large mode (400 tokens). The practical benefit is a seamless UX where 90% of documents (slides, simple forms, short articles) are processed instantly at minimal compute cost, while the remaining 10% receive additional compute as needed — all from a single model with no architectural switching.

Accessibility applications for low-resource language documents. The paper's model flywheel approach for minority language data (Section 3.4.1) — using fitz to create small patch data, training a GOT-OCR2.0 model, then using it to label full documents — demonstrates a template for rapidly bootstrapping OCR for languages where no commercial OCR exists. Combined with DeepSeek-OCR's support for ~100 languages and its low computational requirements (570M active parameters), an accessibility organization could: (1) collect a few thousand pages of scanned documents in a low-resource language, (2) follow the flywheel recipe to generate training data, (3) fine-tune DeepSeek-OCR on this data, (4) deploy on inexpensive hardware (a single A100-40G or even a high-end consumer GPU) to digitize archives, enable screen readers, or provide real-time text-to-speech for printed materials. The practical benefit is reducing the barrier to OCR deployment from "requires a team of ML engineers and a cluster of GPUs" to "requires one motivated engineer with the paper's codebase and a modest compute budget."

When to Prefer This Method

The paper positions DeepSeek-OCR's compression-first architecture against three families of existing vision encoders (Figure 2) and against both pipeline and end-to-end OCR systems (Table 3). The tradeoff is essentially compression ratio vs. formula parsing accuracy, with a secondary tradeoff of end-to-end simplicity vs. marginal accuracy on the hardest documents. The decision rule is:

Prefer DeepSeek-OCR (or the compression-first architectural principle) when:

The deployment bottleneck is throughput (pages/day) or context length (total vision tokens in the LLM's window), not absolute accuracy on mathematically dense content. The 15–65× token reduction directly translates to throughput and context length gains.
The document distribution contains substantial "easy" or "medium" content (slides, books, reports, single-column articles) where 64–256 tokens suffice — these documents constitute the majority of most real-world corpora.
Operational simplicity matters: a single end-to-end model is preferable to maintaining a multi-stage detection→recognition→assembly pipeline.
The use case involves large-scale data generation (millions of pages) where even small per-page token savings compound massively.

Prefer higher-token-count encoders (InternVL, Qwen-VL style) or pipeline systems when:

Formula parsing is a first-class requirement and accuracy below 0.10 edit distance is necessary. DeepSeek-OCR's formula edit distance (0.242–0.277) is 5–10× worse than the best pipeline systems (MinerU2.0: 0.045; dots.ocr at 200dpi: 0.032).
The document distribution is overwhelmingly dense newspapers, multi-column academic papers with heavy mathematical notation, or other layouts where the compression ratio would exceed 10× at the deployed token budget (Table 4 shows newspapers require 795+ tokens for acceptable performance).
The task requires general visual understanding beyond document parsing (complex VQA, spatial reasoning about image content), and the model must handle these tasks without prompt engineering. DeepSeek-OCR is not chat-tuned and requires specific prompts for different capabilities.