Qwen3-VL Technical Report

ArXiv: 2511.21631

🎯 Pitch

Qwen3-VL’s new flagship MoE model scores 85.8 on MathVista while surpassing its text-only backbone on most language benchmarks—inverting the classic VLM trade-off where multimodal training degrades text performance.

---

1. Executive Summary

This paper introduces Qwen3-VL, the most capable vision–language model in the Qwen series, demonstrating superior performance across multimodal benchmarks by integrating three architectural upgrades: Interleaved MRoPE for balanced spatial–temporal positional encoding (distributing temporal, horizontal, and vertical frequency components uniformly across embedding dimensions rather than chunking them into separate subspaces), DeepStack for multi-level vision–language alignment (injecting visual tokens from intermediate ViT layers into corresponding early LLM layers via lightweight residual connections), and text-based timestamp tokens for precise video temporal grounding (prefixing video patches with formatted time strings like <3.0 seconds> instead of encoding absolute time through positional IDs). The flagship Qwen3-VL-235B-A22B achieves state-of-the-art results across reasoning benchmarks—scoring 85.8 on MathVistamini, 74.6 on MathVision, and 72.2 on LogicVista—while the compute-optimal strategy of allocating additional post-training compute and applying square-root reweighting to balance text and multimodal objectives enables Qwen3-VL to surpass its text-only backbone on the majority of language benchmarks, establishing that multimodal training need not erode linguistic proficiency when the training mixture and optimization are carefully calibrated.

2. Context and Motivation

The Core Problem: Multimodal Training Erodes Language Capability—and Prior VLMs Couldn't Stop It

The central tension this paper confronts is one that has plagued vision–language model development from the beginning: when you teach a strong language model to process images and video, its text-only performance almost always degrades. This isn't a minor regression—it's a fundamental trade-off that has forced practitioners to choose between models that are good at language or good at vision, but rarely both simultaneously. The Qwen3-VL team frames this explicitly in Section 1:

"Crucially, these advances must not erode the underlying large language model's (LLM's) linguistic proficiency; multimodal models are expected to match or surpass their text-only counterparts on language benchmarks."

This expectation—that a VLM should match or beat its text-only backbone on pure language tasks—has been aspirational rather than achieved in most prior work. Models that add vision capabilities typically see drops on benchmarks like MMLU, GPQA, and coding tasks because the multimodal training data dilutes the linguistic signal, the vision–language alignment training interferes with learned text representations, or the training objective over-emphasizes visual grounding at the expense of linguistic reasoning.

Why does this matter? The answer has both practical and theoretical dimensions. Practically, if every multimodal training run produces a model that is worse at language than the LLM it started from, then deploying VLMs in production means accepting a regression on text-heavy tasks—document understanding, code generation, mathematical reasoning expressed purely in LaTeX, or any workflow where images are occasionally absent. You'd need to maintain two separate model deployments: one for text, one for multimodal inputs. This doubles infrastructure cost and introduces routing complexity. Theoretically, the erosion of language capability during multimodal training suggests that current approaches to cross-modal alignment are fundamentally lossy—they aren't truly integrating modalities but rather trading one for another. A model that can genuinely integrate vision and language should be better at reasoning than a text-only model, since many concepts (spatial relationships, physical affordances, diagrammatic reasoning) are grounded in visual experience that text alone can only approximate.

The Qwen3-VL team's own results in Tables 5–10 of the evaluation section demonstrate that they have, for the first time at scale, achieved this goal: Qwen3-VL-235B-A22B-Instruct matches or surpasses Qwen3-235B-A22B-Instruct-2507 (the text-only counterpart) on most text-centric benchmarks, and in some cases—AIME-25 mathematics (74.7 vs. 70.3), HMMT-25 (57.4 vs. 55.4), LiveCodeBench v6 (54.3 vs. 51.8)—it substantially outperforms the text-only model. This isn't a claimed result buried in a single table; it's replicated across the 32B and 30B-A3B variants (Tables 7–8), where the multimodal versions consistently outperform their text-only counterparts on reasoning-heavy tasks like AIME-25 (e.g., Qwen3-VL-32B-Instruct at 66.2 vs. Qwen3-32B-Instruct at 20.2—a staggering 3.3× improvement). This reversal of the typical degradation pattern is the paper's most significant practical contribution.

Three Architectural Gaps in Prior VLMs

Beyond the language erosion problem, Qwen3-VL identifies three specific technical limitations in prior vision–language architectures, each of which constrains a different capability dimension.

Gap 1: Imbalanced Positional Encoding for Video

The prior Qwen2.5-VL model (Bai et al., 2025) introduced MRoPE (Multimodal Rotary Position Embedding), which partitioned the embedding dimensions into separate temporal (t), horizontal (h), and vertical (w) subspaces, each assigned distinct rotary frequencies. The intuition was clean: different axes of variation in visual data (time, width, height) should have their own positional encoding subspaces, allowing the model to learn axis-specific positional relationships.

However, the Qwen3-VL team identified a subtle but consequential flaw in this design (Section 2.1). When embedding dimensions are chunked into contiguous t, h, and w blocks, the frequency spectrum becomes imbalanced. The temporal subspace receives only a narrow slice of the total frequency bandwidth, meaning low-frequency temporal patterns (e.g., slow scene changes over many seconds) and high-frequency temporal patterns (e.g., rapid motion between adjacent frames) must both be encoded in a compressed frequency range, reducing discriminability. This is particularly damaging for long videos, where temporal dynamics span multiple timescales—from frame-to-frame motion (milliseconds) to narrative arcs (minutes). The paper explicitly states:

"This results in an imbalanced frequency spectrum, which subsequent studies have shown to degrade performance on long-video understanding benchmarks."

The problem isn't just theoretical. In Qwen2.5-VL, the temporal position IDs for long videos could become excessively large and sparse, since temporal position was directly tied to absolute time via the time-synchronized MRoPE variant. This made the positional encoding less informative for distinguishing nearby temporal positions in long sequences—the same issue that plagued early absolute position embeddings in text transformers before RoPE provided a more graceful solution.

Gap 2: Shallow Vision–Language Fusion Through a Single Projection Layer

In the standard VLM architecture inherited from Qwen2-VL and Qwen2.5-VL, visual information flows into the language model through a single bottleneck: the vision encoder produces a set of visual tokens, an MLP-based merger compresses them (2×2 spatial pooling into a single token), and these merged tokens are prepended to the text sequence at the LLM's input layer. This means all visual information must be compressed into tokens that the LLM's first layer can digest, and all subsequent layers only see whatever representation the first layer managed to extract.

This is a fundamental constraint. Lower layers of the vision encoder capture fine-grained, low-level features—edges, textures, small object details. Higher layers capture semantic, abstract representations—object categories, scene types, relationships. When only the final-layer ViT output is fed to the LLM, the model can only access the highest-level visual abstractions. Any task requiring fine-grained visual discrimination (reading small text in a document, counting objects, identifying subtle visual differences) must rely on representations that have been aggressively pooled and abstracted.

DeepStack (Meng et al., 2024), which the paper cites as inspiration, originally addressed a different problem: stacking tokens from multi-scale visual inputs (different image resolutions) into the LLM. Qwen3-VL extends this idea to extract features from intermediate layers of the vision encoder, not just different input resolutions. This means features from three different ViT depths (early, middle, late) are projected via separate merger modules and added to the hidden states of the first three LLM layers respectively. The early features carry low-level detail that the LLM's later layers can utilize for fine-grained perception; the late features carry semantic information useful for high-level reasoning. This multi-level injection is designed to create a richer, more information-dense interface between vision and language.

Gap 3: Implicit Time Encoding Limits Temporal Grounding Precision

Qwen2.5-VL used a time-synchronized variant of MRoPE to encode temporal information: the temporal position ID for each video frame was directly derived from its absolute timestamp, scaled, and encoded through the rotary position embedding. The Qwen3-VL team identifies two problems with this approach (Section 2.3):

1. Long-video degradation: For long videos, absolute timestamps produce very large temporal position IDs. These large IDs push RoPE's rotary frequencies into regimes where the encoding becomes less discriminative (adjacent time points become harder to distinguish), degrading the model's ability to reason about fine-grained temporal relationships in long videos.

2. Training data inefficiency: To learn the mapping from absolute position IDs to temporal understanding, the model needs to see uniformly sampled examples across a wide range of frame rates and video durations. This imposes a heavy burden on training data construction—you can't just throw videos at the model; you need to carefully control the temporal sampling distribution so the model sees a representative range of position ID values.

The proposed solution is deceptively simple: instead of encoding time through positional embeddings, just write the timestamp as text. Each video temporal patch is prefixed with a formatted string like <3.0 seconds> or 00:01:30 (HMS format), making temporal information directly human-readable in the token stream. This trades a modest increase in context length (the timestamp tokens occupy a few positions) for dramatically more precise and learnable temporal representations, since the model can treat timestamps like any other textual metadata and learn to attend to them appropriately.

Prior Approaches and Their Limitations

Qwen3-VL doesn't emerge from a vacuum. The paper positions itself against a landscape of rapidly evolving VLMs, each with specific strengths but also clear limitations that Qwen3-VL aims to overcome.

Qwen2.5-VL (Bai et al., 2025) is the direct predecessor and the most important baseline. It introduced MRoPE for unified positional encoding across text and vision, dynamic resolution vision encoding, and time-synchronized video understanding. Its limitations—imbalanced MRoPE frequency spectrum, single-level vision fusion, and implicit time encoding—are precisely the three architectural gaps Qwen3-VL addresses. Critically, Qwen2.5-VL-72B was the previous generation's flagship, and Qwen3-VL's medium-sized 32B model already surpasses it on reasoning tasks (Section 5.2), demonstrating that architectural improvements compound with training improvements to create a genuine generational leap.

Gemini 2.5 Pro (Comanici et al., 2025) and GPT-5 (OpenAI, 2025) represent the closed-source frontier. Both offer thinking modes (extended reasoning at inference time) and achieve strong results on multimodal benchmarks. However, they are black boxes: their architectures, training data compositions, and failure modes are opaque. The paper's extensive head-to-head comparisons in Tables 2–4 show Qwen3-VL-235B-A22B competing at parity or better on many benchmarks while remaining fully open-source (Apache 2.0 license). On MMBench, for instance, Qwen3-VL-235B-A22B-Instruct achieves 89.3 EN / 88.9 CN vs. Gemini 2.5 Pro's 88.4 / 86.4 (both non-thinking). On HallusionBench (hallucination detection), Qwen3-VL-235B-A22B-Thinking surpasses Gemini 2.5 Pro by 3.0 points (66.7 vs. 63.7).

Claude Opus 4.1 (Anthropic, 2025) represents the safety-and-alignment-focused alternative. While strong on instruction following and harmlessness, it typically trails on raw reasoning benchmarks. The paper's results confirm this: on MathVistamini, Qwen3-VL-235B-A22B-Thinking reaches 85.8 vs. Opus 4.1's 74.5; on MMMU, 80.6 vs. 74.4 (non-thinking). The gap closes somewhat on subjective tasks (MM-MT-Bench: 8.5 vs. 7.9), but for STEM reasoning, Qwen3-VL consistently leads.

DeepSeek V3 (0324 version) is a strong open-weight text-only model that the paper uses as a comparison point for text-centric tasks (Table 5). Qwen3-VL-235B-A22B-Instruct outperforms it on most reasoning benchmarks while additionally handling multimodal inputs, reinforcing the "no language erosion" claim.

Smaller open-source VLMs like Pixtral 12B (Agrawal et al., 2024) and the Molmo/PixMo family (Deitke et al., 2024) provide competitive baselines at smaller scales, but the paper's results consistently favor Qwen3-VL variants at comparable sizes. The 8B model, for instance, achieves 75.3 on MMStar (thinking mode) and 81.4 on MathVistamini, figures that would have been considered large-model territory just a generation ago.

Beyond specific competing models, the paper identifies a systematic gap in the evaluation landscape that it aims to fill. Prior VLM evaluations tended to focus on narrow capability slices—VQA accuracy, caption quality, or reasoning on curated benchmarks. Qwen3-VL's evaluation (Section 5) spans an unusually broad spectrum: 2D/3D grounding, embodied spatial reasoning, GUI agent tasks, video temporal grounding, multilingual OCR across 39 languages, and text-centric benchmarks from knowledge to coding to alignment. This breadth reflects the paper's ambition: Qwen3-VL isn't optimized for a single benchmark leaderboard but positioned as a general-purpose multimodal foundation model suitable as a backbone for diverse downstream applications.

How Qwen3-VL Positions Itself

The paper's positioning is most clearly articulated through its three-pillar framework stated in the abstract and demonstrated throughout:

1. Stronger pure-text understanding than comparable text-only backbones. This inverts the typical VLM narrative. Rather than apologizing for language degradation as an unavoidable cost of multimodality, Qwen3-VL claims that proper training recipe design (square-root reweighting, careful data mixture, strong-to-weak distillation) can make multimodality a net positive for text reasoning.

2. Robust long-context comprehension with a native 256K-token window. This pushes beyond the 32K–128K context windows common in prior VLMs, enabling the model to process entire textbooks, multi-hour videos, or hundreds of document pages in a single forward pass. The needle-in-a-haystack experiment (Figure 3) demonstrates 99.5% retrieval accuracy at 1M tokens via YaRN-based positional extrapolation, showing the architecture has room to grow beyond the trained context length.

3. Advanced multimodal reasoning across single-image, multi-image, and video tasks. Rather than treating these as separate capabilities requiring different model branches, Qwen3-VL's unified interleaved architecture handles all three modalities through the same attention mechanism, with the interleaved MRoPE providing consistent positional encoding across formats.

The paper also positions itself through a compute allocation philosophy that departs from the "scale pretraining at all costs" approach. Section 4.1 notes:

"Furthermore, we allocate additional compute resources to the post-training phase to further enhance model performance."

This is significant because it reflects a strategic bet: that post-training (SFT, distillation, RL) is under-explored relative to pretraining as a lever for capability improvement. The three-stage post-training pipeline—supervised fine-tuning on long chain-of-thought data, strong-to-weak distillation from larger teacher models, and reinforcement learning across both reasoning and general domains—represents a substantial investment in refining the model after the pretraining foundation is laid. This investment pays off in the competitive results against models that may have had larger pretraining budgets but less sophisticated post-training.

Finally, the paper positions Qwen3-VL as an infrastructure for agents, not just a perception-reasoning engine. The training data includes GUI interaction trajectories, tool-use demonstrations, and multi-turn function-calling sequences. The evaluation benchmarks include OSWorld, AndroidWorld, and ScreenSpot Pro—real interactive environments where the model must plan, click, and adapt. This agent-centric framing looks forward to a world where VLMs aren't just asked "what's in this image?" but "given what you see, what should you do next, and what tool should you use to do it?" The "thinking with images" capability (Section 4.5), where the model can invoke visual processing tools and incorporate their outputs into its reasoning chain, is a concrete step toward this vision.

3. Technical Approach

3.1 Reader Orientation

Qwen3-VL is a vision-language model (VLM) that takes text, images, and video as input and outputs text — the system is a single end-to-end neural network built from three connected components: a vision encoder, a projection module, and a large language model backbone. The core problem it solves is how to add robust visual understanding to a strong language model without degrading its text-only reasoning capabilities, and the shape of the solution is a family of architectural upgrades (interleaved positional encoding, multi-level visual feature injection, explicit text timestamps) combined with a carefully orchestrated four-stage pretraining and three-stage post-training pipeline that balances vision-language and text-only learning objectives through square-root reweighting.

3.2 Big-Picture Architecture (Diagram in Words)

The Qwen3-VL system has three major structural components, which are connected in a feedforward pipeline:

1. Vision Encoder (SigLIP-2-based ViT): Takes raw images or video frames at dynamic, native resolutions and produces a sequence of visual feature tokens. It operates at multiple resolutions, with 2D-RoPE positional encoding and absolute position interpolation based on input size. For Qwen3-VL, an enhanced variant called Qwen3-ViT is used, which is further trained on the SigLIP-2 checkpoint with dynamic resolutions.

2. MLP-based Vision-Language Merger: A two-layer MLP that compresses 2×2 spatial blocks of visual features from the vision encoder into single tokens aligned with the LLM's hidden dimension. There are multiple merger modules — one for the final ViT output (the standard path) and additional mergers for the DeepStack mechanism that projects intermediate ViT layer features into early LLM layers.

3. Large Language Model (Qwen3 backbone): The decoder-only transformer that processes the interleaved sequence of visual tokens and text tokens, with the interleaved MRoPE providing unified positional encoding, and textual timestamp tokens marking temporal positions in video. The model comes in dense variants (2B, 4B, 8B, 32B) and MoE variants (30B-A3B: 30B total, 3B activated per token; 235B-A22B: 235B total, 22B activated per token).

Information flows as follows: images/video frames enter the vision encoder → the vision encoder produces multi-level feature maps at different ViT depths → the merger(s) project these features to the LLM's hidden dimension, optionally routing intermediate features to early LLM layers via DeepStack → the resulting visual tokens are interleaved with text tokens into a unified sequence → the LLM processes the entire sequence with interleaved MRoPE for positional encoding and text-based timestamps for video temporal grounding → the LLM autoregressively generates text output.

3.3 Roadmap for the Deep Dive

• First, the interleaved MRoPE positional encoding mechanism (Section 2.1), since it is the foundational representation that all inputs — text, image, and video — share, and understanding why the original MRoPE's chunked design fails on long video is prerequisite for appreciating the fix.
• Second, the DeepStack multi-level vision-language fusion (Section 2.2), because it explains how rich visual information flows from the vision encoder into the LLM at multiple granularities, which is critical for the model's fine-grained perception capabilities.
• Third, the text-based video timestamp mechanism (Section 2.3), which replaces the implicit temporal encoding of Qwen2.5-VL with explicit text tokens and resolves the long-video degradation problem.
• Fourth, the four-stage pretraining recipe (Section 3.1), which progressively builds capability from basic vision-language alignment through to ultra-long-context adaptation, and the square-root reweighting optimization that balances text and multimodal objectives.
• Fifth, the three-stage post-training pipeline (Section 4.1) — SFT, strong-to-weak distillation, and reinforcement learning — along with the cold-start data construction strategies that enable thinking and non-thinking model variants.
• Sixth, the "thinking with images" agentic capability (Section 4.5), which extends the model from passive perception to active visual tool use through multi-turn, tool-integrated reinforcement learning.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and training methodology paper whose core idea is that three architectural innovations — interleaved MRoPE, DeepStack, and text-based video timestamps — when combined with a carefully calibrated four-stage pretraining pipeline and an extensive three-stage post-training process that allocates additional compute to post-training phases, can produce a VLM that not only matches but outperforms its text-only backbone on language benchmarks while achieving state-of-the-art multimodal reasoning.

---

Interleaved MRoPE: Fixing the Frequency Imbalance in Multimodal Positional Encoding

The Original MRoPE and Its Problem

MRoPE (Multimodal Rotary Position Embedding), introduced in Qwen2-VL (Wang et al., 2024c), was designed to encode positional information for sequences that mix text, images, and video. The key insight was that different modalities have different positional axes: text has a single sequential position (token index), images have two spatial axes (horizontal and vertical position within the 2D grid), and video adds a temporal axis (frame index over time). MRoPE partitions the embedding dimensions — the $d$-dimensional vector that represents the position of each token — into three contiguous chunks:

• A temporal (t) subspace, occupying some contiguous range of embedding dimensions (say, the first $d_t$ dimensions), which encodes the frame index or time position of a video token.
• A horizontal (h) subspace, occupying the next $d_h$ dimensions, which encodes the x-coordinate within the image or frame.
• A vertical (w) subspace, occupying the remaining $d_w$ dimensions, which encodes the y-coordinate.

Each subspace is assigned its own rotary frequencies, using the standard RoPE mechanism: for dimension index $i$ in a given subspace, the rotary frequency is $\theta_i = 10000^{-2i/d_{\text{subspace}}}$, where $d_{\text{subspace}}$ is the number of dimensions in that subspace. The idea was that the temporal, horizontal, and vertical relationships would be learned separately in their dedicated sub-spaces, and the model would naturally combine them through attention.

The Qwen3-VL team identified a critical flaw: this chunked allocation produces an imbalanced frequency spectrum. Because the temporal subspace receives only a fraction of the total embedding dimensions, its rotary frequencies are compressed into a narrow range. Specifically, if $d_t$ is small relative to $d$, the frequencies assigned to temporal positions span a smaller range of angular velocities than the frequencies assigned to spatial positions. This means the temporal encoding has poor resolution: it cannot simultaneously represent very fast temporal changes (frame-to-frame motion, requiring high-frequency encodings that vary rapidly with small position changes) and very slow temporal changes (narrative arcs spanning minutes, requiring low-frequency encodings that vary slowly with large position changes). The paper states this explicitly:

"In its original formulation, the embedding dimensions are partitioned into temporal (t), horizontal (h), and vertical (w) subspaces, each assigned distinct rotary frequencies. This results in an imbalanced frequency spectrum, which subsequent studies have shown to degrade performance on long-video understanding benchmarks."

The mechanism of degradation is subtle. RoPE works by applying a rotation to each pair of embedding dimensions, where the rotation angle is proportional to the position index times a frequency that depends on the dimension index. Low-frequency dimensions (small rotation per position step) encode long-range dependencies; high-frequency dimensions (large rotation per position step) encode short-range dependencies. When all temporal information is compressed into a small $d_t$, there are simply fewer distinct frequencies available for temporal differentiation. Two frames that are 100 positions apart in time might receive very similar temporal encodings if the available frequencies can't distinguish that gap, making it hard for the model to attend to specific temporal moments in a long video.

Additionally, for text-only tokens (which have no spatial or temporal meaning), MRoPE sets the t, h, and w position IDs to the same value — the token's sequential position in the text. This means the text encoding is effectively a standard RoPE with the full frequency range, but the video temporal encoding operates with only a fraction of that range. The asymmetry creates a representation gap: temporal relationships in video are encoded less richly than sequential relationships in text, even though both are fundamentally about ordering.

The Interleaved MRoPE Solution

The fix is elegant and directly addresses the root cause. Instead of grouping all temporal dimensions together in a contiguous block, interleaved MRoPE distributes the t, h, and w components uniformly across the embedding dimensions. The paper describes it as follows:

"We redesign the frequency allocation by interleaving the t, h, and w components across the embedding dimensions (Huang et al., 2025). This ensures that each spatial–temporal axis is uniformly represented across both low- and high-frequency bands. The resulting balanced spectrum mitigates the original spectral bias and significantly improves long-range positional modeling for video."

Concretely, rather than assigning dimensions $1$ through $d_t$ to temporal, $d_t + 1$ through $d_t + d_h$ to horizontal, and the rest to vertical, the interleaved design assigns dimension $1$ to temporal, dimension $2$ to horizontal, dimension $3$ to vertical, dimension $4$ to temporal, dimension $5$ to horizontal, and so on, cycling through t, h, w. Each axis now spans the full frequency range — from the lowest-frequency dimension (with the largest rotary period) to the highest-frequency dimension (with the smallest rotary period) — because each axis gets dimensions at every position in the frequency ordering.

To make this concrete with numbers: suppose the embedding dimension is $d = 1152$ (a typical transformer hidden size). In the original MRoPE with equal partitioning, temporal would get dimensions $1$–$384$, horizontal would get $385$–$768$, and vertical would get $769$–$1152$. The highest-frequency temporal dimension would be dimension $384$, with frequency $\theta_{384} = 10000^{-2 \cdot 384 / 384} = 10000^{-2} \approx 10^{-8}$, while the highest-frequency horizontal dimension would be dimension $768$, with frequency $\theta_{768} = 10000^{-2 \cdot 768 / 384} = 10000^{-4} \approx 10^{-16}$ — a 100-million-fold difference in the finest resolution available to each axis. In the interleaved design, each axis gets dimensions at every position in $1$ through $1152$, so all three axes share the same frequency range, with each axis having both low-frequency dimensions (for long-range dependencies) and high-frequency dimensions (for fine-grained discrimination).

What This Means for the Model's Attention Patterns

The practical effect is that when the model computes attention scores between two video tokens, the dot product now involves contributions from temporal, horizontal, and vertical position differences at every frequency band. The attention score between token $a$ at position $(t_a, h_a, w_a)$ and token $b$ at position $(t_b, h_b, w_b)$ is computed as a sum over all embedding dimensions, and each dimension $i$ contributes a term that depends on:

• If $i$ is a temporal dimension: $\cos((t_a - t_b) \cdot \theta_i)$
• If $i$ is a horizontal dimension: $\cos((h_a - h_b) \cdot \theta_i)$
• If $i$ is a vertical dimension: $\cos((w_a - w_b) \cdot \theta_i)$

Since each axis now has dimensions spanning the full spectrum of $\theta_i$ values, attention can be simultaneously sensitive to small temporal gaps (via high-frequency temporal dimensions), large temporal gaps (via low-frequency temporal dimensions), and similarly for spatial gaps. This is particularly important for video tasks like temporal action localization, where the model needs to attend to a specific short time window (high temporal frequency) within a long video context (low temporal frequency).

For text tokens, interleaved MRoPE still sets all three position IDs to the same sequential index, but now the encoding is distributed across all dimensions uniformly, making the text encodings structurally similar to standard RoPE but with the additional capacity to generalize to multimodal inputs.

Why This Design Choice Over Alternatives

The paper doesn't discuss alternative fixes to the MRoPE frequency imbalance, but we can infer the reasoning. One alternative would be to simply increase $d_t$, giving temporal a larger share of the embedding dimensions. This would mitigate the frequency range compression but at the cost of reducing spatial encoding capacity, and it doesn't solve the fundamental problem that different axes shouldn't be segregated into separate frequency bands. Another alternative would be to use separate positional encodings for each modality and concatenate or add them, but this loses the unified representation that MRoPE was designed to provide — vision and text tokens in the same sequence should use the same positional encoding scheme so the model can learn cross-modal attention patterns. The interleaved design preserves unity while fixing the frequency balance, making it the minimal intervention that addresses the root cause.

---

DeepStack: Multi-Level Visual Feature Injection into the Language Model

The Limitation of Single-Point Vision-Language Fusion

In standard VLM architectures, the vision encoder produces one set of output tokens (typically after the final transformer layer of the ViT), and an MLP-based merger projects these tokens to the LLM's hidden dimension. These projected visual tokens are then prepended to the text token sequence and fed as input to the first layer of the LLM. This means all subsequent LLM layers only have access to whatever visual information survived the compression through the final ViT layer and the merger.

The final layer of a Vision Transformer has undergone many rounds of self-attention and feedforward processing. By this point, the representations have become abstract and semantic — excellent for recognizing "this is a dog" or "this is a kitchen scene," but potentially missing fine-grained details that were present in earlier layers: precise edge locations, texture patterns, small text, or subtle color gradients. Early ViT layers tend to capture local features (edges, corners, textures); middle layers capture mid-level features (object parts, shapes, patterns); late layers capture high-level semantic features (object categories, scene types, relationships). When only the late-layer output is fed to the LLM, the model is being asked to do fine-grained visual reasoning (e.g., "read the small text on this sign," "count the number of people in this crowd") using representations that have been optimized for semantic classification, not spatial precision.

The paper's insight is that this is an unnecessary bottleneck. The ViT already computes rich intermediate representations at every layer — why throw them away? DeepStack (Meng et al., 2024) originally addressed a related problem by feeding visual tokens from multiple image resolutions into the LLM. Qwen3-VL extends this idea to feed visual tokens from multiple depths of the ViT.

The DeepStack Mechanism in Qwen3-VL

The implementation (Section 2.2, Figure 1) works as follows:

1. The vision encoder is a standard ViT with $L$ layers (the exact number depends on the SigLIP-2 variant — 400M parameters for the main model, 300M for the 2B and 4B variants). Each layer produces an intermediate representation of the image, with the same spatial resolution (number of patches) but increasingly abstract features.

2. Features are extracted from three distinct levels of the vision encoder. The paper doesn't specify the exact layer indices, but the natural interpretation is: early layers (e.g., layer $\approx L/4$), middle layers (e.g., layer $\approx L/2$), and the final layer (layer $L$). These three levels correspond roughly to low-level, mid-level, and high-level visual features.

3. Dedicated merger modules — separate two-layer MLPs — project each level's features to the LLM's hidden dimension. The standard merger for the final ViT output still exists; the DeepStack mergers are additional, parallel projection paths for the earlier features.

4. The projected tokens from each ViT level are added directly to the hidden states of the corresponding early LLM layers. Specifically: early ViT features go to LLM layer 1, middle ViT features go to LLM layer 2, and late ViT features go to LLM layer 3 (or equivalently, to the input of the LLM, which is the standard path). The paper states:

"Subsequently, dedicated vision–language merger modules project these multi-level features into visual tokens, which are then added directly to the corresponding hidden states of the first three LLM layers."

This is a residual connection: the projected visual tokens are summed with the existing hidden state of that LLM layer, so the layer can choose to use or ignore the injected visual information via its subsequent processing. The residual addition means the visual tokens from different ViT depths are not concatenated (which would increase sequence length and memory usage) but rather fused into the representation at the point where they're most useful.

What This Enables: Cross-Layer Visual-Textual Interaction

The benefit of this design is that different levels of visual information become available at different depths of the LLM's processing. Early LLM layers, which tend to handle local syntactic and surface-level features in text, now also have access to low-level visual features — edges, textures, fine spatial details. This is important for tasks like OCR, where the model needs to distinguish subtle shape differences between characters, or grounding, where the model needs to identify precise object boundaries. Middle LLM layers, which handle more compositional reasoning, have access to mid-level visual features — object parts, shapes, configurations. Late LLM layers, which handle abstract reasoning, have access to high-level visual features — semantic categories, scene understanding.

The paper's ablation study (Table 12) quantifies the benefit: switching from the baseline (no DeepStack) to DeepStack improves the average accuracy across 11 benchmarks from 74.7 to 76.0, with particularly large gains on fine-grained perception tasks. InfoVQA (which requires reading text in infographics) improves from 71.9 to 74.2 (+2.3 points), DocVQA (document question answering) from 89.5 to 91.1 (+1.6 points), and AI2D (diagram understanding) from 81.8 to 83.2 (+1.4 points). These are all tasks where access to low-level visual detail — small text, fine lines, precise spatial relationships — is critical. The gains on more semantic tasks like MMBench-EN (81.0 to 81.2, +0.2) and TVQA (80.6 to 80.5, -0.1) are negligible, consistent with the interpretation that DeepStack primarily helps with fine-grained perception rather than high-level understanding.

A Key Difference from the Original DeepStack

The paper notes an important distinction from the original DeepStack work:

"Unlike the original DeepStack approach, which stacks tokens from multi-scale visual inputs, we extend DeepStack to extract visual tokens from intermediate layers of the Vision Transformer (ViT)."

The original DeepStack was about multi-resolution fusion: take the same image at different resolutions, run each through the ViT, and inject the resulting tokens at different LLM layers. Qwen3-VL's innovation is to get multi-level information from a single forward pass through the ViT, by tapping into intermediate layers. This is more computationally efficient (no need for multiple ViT forward passes) and potentially more informative, since ViT intermediate layers contain different types of features than multi-resolution inputs would (intermediate layers capture abstraction level, while multi-resolution inputs capture spatial scale — these are related but distinct axes of variation).

---

Text-Based Video Timestamps: Explicit Temporal Grounding

The Problem with Implicit Time Encoding

Qwen2.5-VL used a time-synchronized variant of MRoPE to give the model temporal awareness. The idea was straightforward: for each video token, instead of using a simple sequential frame index as the temporal position ID, use the absolute timestamp (in seconds) scaled and mapped to an integer position ID. This meant the temporal rotary encoding was directly tied to physical time — a token at 3.5 seconds into the video would have a different temporal encoding than a token at 3.6 seconds, proportional to the 0.1-second gap.

The paper identifies two problems (Section 2.3):

1. Sparse and large temporal position IDs for long videos. For a 2-hour video at 1 fps, the timestamp at the end would be 7200 seconds. After scaling and quantization, this produces very large position ID values. RoPE's rotary frequencies are designed around a range of position IDs roughly matching the training context length — typically up to a few hundred thousand at most. When position IDs become very large (millions), the rotary encoding loses discriminability because the angular differences between nearby positions become submerged in the overall rotation. Two frames at 7199 and 7200 seconds become nearly indistinguishable from two frames at 0 and 1 second, because the rotary encoding's periodicity wraps around. This makes it hard for the model to perform fine-grained temporal localization in long videos.

2. Training data coverage requirements. To learn a meaningful mapping from timestamp-derived position IDs to temporal understanding, the model needs to see a representative distribution of timestamp values during training. This means the training data must include videos with diverse frame rates, durations, and temporal sampling patterns, so the model encounters a wide range of position ID values. The paper notes this "significantly increases the cost of training data construction" because it's not enough to have long videos — you need them sampled at various frame rates to cover the timestamp space uniformly.

The Text-Based Solution

The solution is disarmingly simple: instead of encoding time through positional embeddings (a continuous, neural representation that the model must learn to interpret), encode time as text (a discrete, symbolic representation that the model already knows how to process). The paper states:

"We adopt a textual token–based time encoding strategy (Chen et al., 2024b), wherein each video temporal patch is prefixed with a timestamp expressed as a formatted text string—e.g., <3.0 seconds>. Furthermore, during training, we generate timestamps in both seconds and HMS (hours:minutes:seconds) formats to ensure the model learns to interpret diverse timecode representations."

Concretely, when processing a video, the model doesn't just receive the visual tokens from each frame. Before the visual tokens for frame $f$, it receives a text token sequence like <3.0 seconds> (or 00:00:03 in HMS format). This timestamp is tokenized by the standard text tokenizer and embedded just like any other text. The model can then attend to timestamps using exactly the same mechanisms it uses for other text — it can learn that <3.0 seconds> and <3.1 seconds> are close in meaning because the text embeddings encode numerical proximity via shared subword tokens and positional patterns in training data.

Advantages of the Text-Based Approach

First, precision: natural language timestamps are inherently precise. The model doesn't need to learn a mapping from a scalar position ID to a time value; the time value is directly stated in a format the model has seen throughout its text training. Second, context length efficiency: a timestamp like <3.0 seconds> is about 3-5 tokens in most tokenizers. For a 1-hour video at 2 fps (7200 frames), this adds 21,600-36,000 tokens — a modest overhead relative to the visual tokens themselves (which at 640 tokens per frame would be 4.6 million tokens). The paper explicitly acknowledges this trade-off: "Although this approach incurs a modest increase in context length, it enables the model to perceive temporal information more effectively and precisely."

Third, robustness to frame rate variation: text timestamps are independent of frame rate. Whether the video is sampled at 1 fps or 30 fps, the timestamp text for the 10-second mark is always <10.0 seconds>. This means the training data doesn't need to cover a uniform distribution of frame rates — the model learns a stable temporal representation that works across sampling regimes.

Fourth, the dual-format training (seconds and HMS) is a clever robustness measure. In real-world applications, timestamps might be expressed in either format depending on the context (e.g., video editing software might use HMS, while scientific applications might use seconds). By training with both, the model learns to recognize temporal references in whichever format they appear, making it more flexible for downstream tasks.

How This Replaces T-RoPE

The paper's description "evolving from T-RoPE to explicit textual timestamp alignment" (abstract) refers to this transition. T-RoPE (the time-synchronized MRoPE variant in Qwen2.5-VL) encoded time as a component of the positional embedding — a low-level, architectural signal. The new approach encodes time as content — a visible, attendable token in the input sequence. This distinction matters because it changes what the model can do with temporal information. With T-RoPE, temporal relationships only influence attention scores through the positional component of the dot product. With text timestamps, temporal relationships can influence both attention (the model can learn to attend to specific timestamp tokens) and the feedforward layers (the model can reason about time in the same way it reasons about any other textual information — e.g., "the event at 3.0 seconds is before the event at 5.0 seconds").

---

Four-Stage Pretraining: Progressive Capability Building

The pretraining recipe is structured into four sequential stages (Table 1), each with a specific objective, training data composition, context length, and parameter update schedule. The progression is designed to build capabilities from the simplest (aligning visual and language representations) to the most complex (processing ultra-long multimodal sequences).

Stage 0: Vision-Language Alignment (67B tokens, sequence length 8,192)

This is the warm-up phase. The objective is to bridge the initial modality gap between the pretrained vision encoder and the pretrained LLM — two models that have never seen each other's outputs and whose representations live in entirely different spaces. The key design decision is that only the MLP merger parameters are trained; both the vision encoder and the LLM backbone remain frozen:

"Crucially, only the parameters of the MLP merger are trained during this phase, while both the vision encoder and the LLM backbone remain frozen."

The training data consists of approximately 67 billion tokens of high-quality image-caption pairs, visual knowledge collections, and OCR data. Image-caption pairs teach the merger to map visual features to the kinds of representations that the LLM expects for descriptive text. OCR data teaches the merger to preserve fine-grained textual information from images. Visual knowledge data teaches the merger to extract semantically meaningful features that correspond to factual knowledge the LLM already possesses in text form.

Why freeze the LLM and vision encoder? If you train all parameters from the start, the randomly initialized merger (or at least, the merger with weights not yet adapted to connect these two specific models) would produce noisy gradients that could disrupt the carefully pretrained representations in both the ViT and the LLM. By training only the merger initially, you allow it to converge to a reasonable mapping function without disturbing the other components. This is analogous to the "linear probe" approach in transfer learning, where you first train a simple classifier on frozen features before fine-tuning the whole network.

The sequence length of 8,192 is the base context window — sufficient for single images with captions and short documents, but not yet extended for long videos or multi-page documents.

Stage 1: Multimodal Pre-Training (~1T tokens, sequence length 8,192)

With the merger providing a reasonable initial mapping, Stage 1 unfreezes all components for full-parameter end-to-end training:

"In this phase, we unfreeze all model components—the vision encoder, the merger, and the LLM—for joint end-to-end training."

The training data expands dramatically to approximately 1 trillion tokens and becomes much more diverse. The VL portion adds interleaved image-text documents (images placed within text at natural positions), visual grounding tasks (locating objects described in text), visual question answering (VQA), STEM domain data (science and math diagrams with explanations), and a small amount of video data to introduce temporal understanding. Crucially, the data mixture includes both vision-language data and text-only data, because the goal is to maintain the LLM's language capabilities while adding visual understanding.

The paper mentions a critical optimization detail here: square-root reweighting. This is the mechanism for balancing text and multimodal learning objectives. In standard training, each token in the sequence contributes equally to the loss. But if most of the training data is vision-language (where visual tokens might dominate the sequence length), the model could over-optimize for visual tasks at the expense of text understanding. Square-root reweighting adjusts the per-token loss based on the type of data. Specifically, for a mini-batch containing both text-only and multimodal examples, the loss for each example is weighted by the inverse square root of its token count (or a related normalization), so that examples with very different lengths contribute more equally to the gradient. The paper's abstract notes this explicitly:

"To balance text-only and multimodal learning objectives, we apply square-root reweighting, which boosts multimodal performance without compromising text capabilities."

The sequence length remains at 8,192, focusing this stage on building strong single-image and short-document capabilities before extending the context window.

Stage 2: Long-Context Pre-Training (~1T tokens, sequence length 32,768)

Stage 2 is where the context window quadruples to 32,768 tokens. This is a substantial jump that requires careful data mixture adjustments:

"Training is conducted on a dataset of approximately 1T tokens, with an adjusted data mixture to support long-context tasks. The proportion of text-only data is increased to bolster long-form text comprehension, while the remaining VL data incorporates a significantly larger volume of video and agent-oriented instruction-following data."

The increased text-only proportion is strategic: long-form text comprehension (understanding books, articles, documentation) is fundamentally a language capability, and maintaining it requires the model to continue learning from long text sequences, not just long multimodal sequences. The video data increases because longer context enables processing more frames from longer videos, providing richer temporal narratives. The agent-oriented instruction-following data — which consists of multi-step task trajectories, tool use demonstrations, and GUI interaction sequences — benefits from the extended context because agent tasks often involve many sequential steps that must be tracked over long contexts.

All model parameters continue to be trainable, allowing the positional encodings (interleaved MRoPE) to adapt to the extended range of position IDs and the attention mechanisms to learn long-range dependencies.

Stage 3: Ultra-Long-Context Adaptation (100B tokens, sequence length 262,144)

The final pretraining stage pushes the context window to its operational maximum: 256K tokens (262,144 = $256 \times 1024$). This is a specialized adaptation phase, not a full training run:

"The final stage (S3) is a specialized phase designed to push the model's context window to its operational limits. Here, we dramatically increase the sequence length to 262,144."

The training data volume is smaller (100B tokens) but carefully curated for long-context tasks: long-video understanding (multi-hour videos requiring thousands of frames) and long-document understanding (entire books, multi-hundred-page PDFs). The data mixture again includes both text-only and VL data, with strong emphasis on tasks that explicitly require processing very long sequences from start to finish.

The paper notes that this final adaptation "solidifies Qwen3-VL's proficiency in processing and analyzing extremely long sequential inputs" — it's not about learning new capabilities but about ensuring the model can reliably use its existing capabilities across the full 256K context range. The needle-in-a-haystack experiment (Section 5.12.3, Figure 3) confirms this: the model achieves 100% accuracy on videos up to 256K tokens (30 minutes) and 99.5% accuracy when extrapolated to 1M tokens (120 minutes) via YaRN-based positional extension, demonstrating that the attention mechanisms learned during Stage 3 generalize even beyond the trained context length.

---

Post-Training Pipeline: SFT, Distillation, and RL

The post-training process (Section 4.1) has three stages that refine the pretrained model into both instruct (non-thinking) and thinking variants. The paper emphasizes that additional compute resources are allocated to post-training, reflecting the philosophy that post-training is an under-explored lever for capability improvement.

Stage 1: Supervised Fine-Tuning (SFT)

The SFT stage imparts instruction-following abilities and activates latent reasoning skills through approximately 1.2 million curated examples. It is conducted in two phases:

• Phase 1 (32K context): One epoch of training at 32K sequence length, covering the full diversity of the SFT dataset.
• Phase 2 (256K context): A second epoch at the full 256K sequence length, with a curriculum that interleaves long-context inputs (hundreds of pages of documents, entire textbooks, videos up to 2 hours) with data sampled at 32K to maintain general capability.

The SFT data is strategically composed: one-third text-only, two-thirds image-text and video-text pairs. This split is designed to maintain language capability while teaching multimodal skills. The paper notes:

"Given Qwen3-VL's native support for a 256K token context length, we employ a staged training strategy to optimize for computational efficiency."

The bifurcation into non-thinking and thinking variants happens at the data level. Non-thinking models are trained on standard question-answer pairs. Thinking models are trained on Chain-of-Thought (CoT) formatted data where the reasoning process is explicitly modeled. The paper states:

"To cater to different needs, we bifurcate the training data into standard formats for non-thinking models and Chain-of-Thought (CoT) formats for thinking models, the latter of which explicitly models the reasoning process."

The data undergoes a rigorous two-phase filtering pipeline. Query Filtering uses Qwen2.5-VL to discard unverifiable or ambiguous queries, minimally revise unclear instructions, eliminate web-sourced queries lacking substance, and assess complexity and relevance. Response Filtering combines rule-based heuristics (removing repetitive, incomplete, off-topic, or harmful responses) with model-based evaluation (using reward models from the Qwen2.5-VL series to score correctness, completeness, clarity, and helpfulness, with special emphasis on visual interpretation accuracy for grounding tasks).

For thinking models, the Long-CoT cold start dataset imposes additional curation: difficulty filtering (retaining only problems where baseline models had low pass rates or generated longer responses), multimodal necessity filtering (discarding vision-language math problems that Qwen3-30B-nothink could solve without visual input), and response quality control (removing incorrect answers and responses with undesirable patterns like excessive repetition or guessing without reasoning).

Stage 2: Strong-to-Weak Distillation

The second stage transfers capabilities from larger, more capable teacher models to the student Qwen3-VL models through knowledge distillation. The paper describes a two-phase distillation pipeline:

• Off-policy Distillation: Outputs generated by teacher models are used to provide response distillation. The student model is fine-tuned on these teacher-generated responses, learning to mimic the teacher's reasoning and output style. This establishes a strong foundation for subsequent training.

• On-policy Distillation: The student model generates its own responses to the prompts. These on-policy sequences are then used for fine-tuning, but with an additional KL divergence loss that aligns the student's predicted token probabilities (logits) with the teacher's predictions. The paper states:

"We align the logits predicted by the student and teacher by minimizing the KL divergence."

This two-phase approach is important. Off-policy distillation alone would train the student to imitate the teacher's outputs, but the student might not learn the underlying reasoning process that generated those outputs — it might just memorize surface patterns. On-policy distillation, by working with the student's own generations, encourages the student to internalize the teacher's reasoning distribution. The KL divergence term provides a soft target: even when the student's exact token choices differ from the teacher's, it is encouraged to maintain a similar probability distribution over tokens, preserving the teacher's uncertainty and reasoning patterns.

Stage 3: Reinforcement Learning

The final stage applies RL to further refine model performance and alignment. It is divided into two sub-stages:

Reasoning RL: This focuses on tasks with deterministic verifiability — mathematics, coding, logical reasoning, visual grounding, and visual puzzles. Solutions can be verified via rules or code executors. The training data is approximately 30K RL queries, carefully curated by:

• Sampling 16 responses per query from a preliminary checkpoint of Qwen3-VL-235B-A22B.
• Discarding any query where all 16 responses are incorrect (these are too hard for the current model to learn from via RL).
• Running preliminary RL experiments per task to identify and remove data sources with limited improvement potential.
• For the final training, sampling 16 responses per query and filtering out easy queries where pass rate exceeds 90%.

The reward system provides task-specific rewards with shared infrastructure (data preprocessing, utility functions, a reward manager). To mitigate code-switching (the model mixing languages inappropriately), a penalty is applied when the response language differs from the prompt language. The RL algorithm is SAPO (Gao et al., 2025), which the paper describes as "a smooth and adaptive policy-gradient method" that "delivers consistent improvements across diverse text and multimodal tasks and across different model sizes and architectures."

General RL: This broader stage enhances generalization and operational robustness. The reward function is formulated based on SFT tasks including VQA, image captioning, OCR, document parsing, grounding, and clock recognition. It optimizes two dimensions:

1. Instruction Following: Adherence to explicit user directives on content, format, length, and structured outputs (e.g., JSON).
2. Preference Alignment: For open-ended queries, alignment with human preferences on helpfulness, factual accuracy, and stylistic appropriateness.

A novel aspect of this stage is its use as a corrective mechanism to unlearn specific errors ingrained during SFT. The paper introduces specialized, verifiable tasks designed to trigger these errors (e.g., counter-intuitive object counting, complex clock time recognition) and uses targeted RL to supplant erroneous priors with factual knowledge. Additionally, to mitigate low-prevalence but persistent issues like language mixing, repetition, and formatting errors, the paper curates a dedicated dataset of prompts known to elicit these behaviors and applies targeted, high-frequency penalties during RL.

The reward system for General RL is hybrid:

• Rule-Based Rewards: For tasks with verifiable ground truth (format adherence, instruction following), well-defined heuristics provide unambiguous, hack-resistant feedback.
• Model-Based Rewards: Qwen2.5-VL-72B-Instruct or Qwen3 serves as a judge, evaluating each generated response against a ground-truth reference across multiple quality axes (correctness, completeness, clarity, helpfulness, visual interpretation accuracy). This approach handles nuanced, open-ended tasks where strict rule-based matching would produce false negatives.

---

Thinking with Images: Agentic Visual Tool Use

Section 4.5 describes a specialized capability that enables Qwen3-VL to actively process visual information through tool calls during reasoning — the model can "zoom in" on image regions, ask for specific visual details, and iteratively refine its understanding.

Two-Stage Training Paradigm

The training proceeds in two stages, each with a similar structure but differing in data source and scale:

Stage 1: Cold-Start Agentic SFT + RL. A dataset of approximately 10K grounding examples is synthesized, consisting of simple two-turn visual question answering tasks (e.g., attribute detection). Qwen2.5-VL-32B is fine-tuned via SFT to emulate the behavior of a visual agent following a think → act → analyze feedback → answer loop. Multi-turn, tool-integrated RL is then applied to further enhance reasoning.

Stage 2: Distillation and Scaling. The trained Qwen2.5-VL-32B visual agents from Stage 1 are used to generate a larger, more diverse dataset of approximately 120K multi-turn agentic interactions spanning a broader range of visual tasks. This distilled data, combined with additional synthesized data, is used for cold-start SFT and tool-integrated RL of Qwen3-VL.

Tool-Integrated RL with Three Reward Signals

During RL, three complementary reward signals encourage robust, tool-mediated reasoning:

1. Answer Accuracy Reward: Qwen3-32B measures whether the final answer is correct. This is the primary task-completion signal.
2. Multi-Turn Reasoning Reward: Qwen2.5-VL-72B evaluates whether the assistant correctly interprets tool or environment feedback and arrives at the answer through coherent, step-by-step reasoning. This prevents the model from arriving at the right answer through luck or shortcuts.
3. Tool-Calling Reward: Encourages appropriate tool usage by comparing the actual number of tool calls to an expert-estimated target (determined offline by Qwen2.5-VL-72B based on task complexity).

The paper explicitly notes a failure mode observed in early experiments:

"Early experiments reveal a tendency for models to degenerate into making only a single tool call to hack the first two rewards, regardless of task demands."

The tool-calling reward is the countermeasure: by rewarding the model for making an appropriate number of tool calls for the task's complexity, it prevents the degenerate strategy of making one tool call and guessing the answer. This is a concrete example of reward hacking in multi-step agentic tasks and demonstrates the need for careful reward design that anticipates and penalizes shortcut behaviors.

4. Key Insights and Innovations

Innovation 1: Multimodal Training as a Net Positive for Language Capability — Reversing the Degradation Assumption

The dominant assumption in vision–language model development has been that adding visual capabilities to a strong language model inevitably erodes its text-only performance — a trade-off practitioners accept as the cost of multimodality. Qwen3-VL's most conceptually significant contribution is empirically demonstrating that this trade-off is not fundamental but rather an artifact of suboptimal training recipes. The paper doesn't just claim to minimize degradation; it shows multimodal models outperforming their text-only counterparts on language benchmarks, sometimes dramatically (Qwen3-VL-32B-Instruct scores 66.2 on AIME-25 mathematics versus 20.2 for Qwen3-32B-Instruct — a 3.3× improvement; Tables 7–8).

This finding reframes the problem entirely. Instead of asking "how do we protect language capability from vision training?" (a defensive posture), the question becomes "how do we design training so that visual grounding enhances linguistic reasoning?" (an integrative posture). The mechanism — square-root reweighting of per-token losses to balance text and multimodal objectives, careful data mixture composition across all pretraining stages, and strong-to-weak distillation that transfers reasoning patterns rather than just surface outputs — suggests that the degradation observed in prior VLMs was not caused by competition between modalities for representational capacity, but by an implicit bias in the training objective that over-weighted multimodal examples and under-weighted text-only learning, causing the model to drift from its language pretraining optimum.

This is a fundamental insight rather than an incremental improvement because it changes the design philosophy for VLM training. Prior work (Qwen2-VL, Qwen2.5-VL, and most other open-source VLMs) accepted some language degradation as inevitable and focused on minimizing it. Qwen3-VL shows that with proper objective balancing, multimodality can be a complementary signal that improves reasoning — perhaps because visual examples of mathematical concepts, spatial relationships, or logical structures provide grounding that text alone cannot. The fact that this pattern holds across model sizes (from 2B to 235B) and architectures (dense and MoE) suggests it is a robust property of the training methodology, not a fluke of scale.

Innovation 2: Interleaved MRoPE as a Diagnostic Fix for a Fundamental Representational Flaw

The interleaved MRoPE design (Section 2.1) appears at first glance to be a minor architectural tweak — rearrange the dimension ordering in the positional encoding — but it represents a deeper conceptual contribution: identifying and fixing a frequency spectrum imbalance that previous work had not diagnosed. The original MRoPE (Qwen2-VL, Qwen2.5-VL) partitioned embedding dimensions into contiguous temporal, horizontal, and vertical blocks, assigning each block its own rotary frequencies. This seemed natural — different axes, different subspaces — but it created a hidden pathology: each axis received only a fraction of the total frequency bandwidth, meaning temporal encoding (for video) operated with far fewer distinct frequencies than spatial encoding, making long-range temporal discrimination fundamentally harder.

The interleaved fix — distributing t, h, and w dimensions uniformly across the full frequency range so each axis spans both low and high rotary frequencies — is elegant precisely because it addresses the root cause rather than patching symptoms. The innovation is less the interleaving mechanism itself and more the diagnostic insight that the chunked design was causing an imbalanced frequency spectrum, a concept that wasn't part of the prior discourse around MRoPE. This is significant beyond raw performance gains on video benchmarks because it provides a general design principle for multimodal positional encoding: when encoding multiple positional axes in a shared embedding space, each axis should have access to the full frequency range to ensure balanced representational capacity across all axes.

The paper's evidence for the impact comes from video understanding benchmarks (Tables 2–4), where Qwen3-VL substantially outperforms Qwen2.5-VL-72B on long-video tasks like MLVU and LVBench, and from the needle-in-a-haystack experiment (Figure 3) showing 99.5% accuracy at 1M tokens. But the deeper significance is that this fix establishes a diagnostic framework for evaluating positional encoding schemes: check whether each positional axis gets uniform coverage of the frequency spectrum, because imbalance produces axis-specific degradation that may not be obvious from aggregate metrics on short-context tasks.

Innovation 3: Explicit Text Timestamps as a Paradigm Shift in Temporal Grounding

The transition from T-RoPE (implicit, positional-encoding-based time representation in Qwen2.5-VL) to text-based timestamps (explicit, tokenized time strings like <3.0 seconds>) in Qwen3-VL represents a paradigm shift in how VLMs represent temporal information. Prior work encoded time as a continuous signal in the positional embedding — a mathematically elegant approach that tied temporal discrimination to the rotary frequency mechanism. Qwen3-VL's key insight is that this elegance came at a cost: for long videos, the temporal position IDs became large and sparse, degrading RoPE's discriminability, and the training data had to be carefully sampled across frame rates to ensure coverage of the timestamp space.

The conceptual move is recognizing that time is better treated as content than as position. By making timestamps visible text tokens, the model can attend to them, reason about them, and relate them using the same mechanisms it uses for any other textual information. This is a fundamental shift because it changes the nature of temporal understanding from a low-level architectural signal (which the model must learn to interpret from scratch during vision–language training) to a high-level semantic signal (which leverages the model's existing linguistic knowledge of numbers and time expressions). The dual-format training (seconds and HMS) reinforces this: the model learns to understand "3.0 seconds" and "00:00:03" as equivalent, demonstrating genuine temporal comprehension rather than just pattern matching against position IDs.

This contribution is significant beyond Qwen3-VL's specific results because it suggests a broader design principle: explicit symbolic representations should be preferred over implicit continuous encodings when the symbolic form is already part of the model's training distribution. Text timestamps work well because the LLM backbone already understands numbers, time, and formatting conventions from its text pretraining. The "modest increase in context length" the paper notes (a few tokens per frame) is a small price for leveraging this existing knowledge. This principle could extend to other modalities — for example, explicit coordinate tokens for spatial grounding rather than implicit position encodings — and the paper's success with timestamps provides strong evidence for this approach.

Innovation 4: DeepStack as Multi-Level Visual Fusion — Extending Cross-Modal Alignment Beyond a Single Bottleneck

The DeepStack mechanism (Section 2.2) extends the original DeepStack idea (Meng et al., 2024) in a conceptually novel direction: rather than injecting visual tokens from multiple input resolutions (the original DeepStack approach), Qwen3-VL injects features from multiple depths of the vision encoder into corresponding early layers of the LLM. The innovation is recognizing that the standard single-bottleneck architecture — where only the final ViT layer's output reaches the LLM — discards rich intermediate representations that are valuable for different types of visual reasoning.

What makes this distinctive at the idea level is the implicit claim about representational complementarity: low-level ViT features (edges, textures, fine spatial details) are most useful for fine-grained perception tasks like OCR and small-object grounding, while high-level ViT features (semantic categories, scene types) are most useful for reasoning tasks like VQA and diagram understanding. By routing these different feature types to different LLM layers (early ViT features to early LLM layers, late ViT features to late LLM layers), DeepStack creates a more granular interface where the LLM can access the right level of visual abstraction for the task at hand.

The ablation study (Table 12) provides evidence for this complementarity claim: DeepStack provides the largest gains on fine-grained perception tasks (InfoVQA +2.3, AI2D +1.4) while having negligible effect on semantic understanding tasks (TVQA -0.1). This specificity is what elevates DeepStack from a generic "more features is better" story to a targeted architectural intervention that addresses a diagnosed limitation. Prior VLMs treated the vision–language interface as a single projection step; Qwen3-VL treats it as a layered information channel where different visual abstraction levels support different downstream capabilities. This is a fundamental architectural insight that changes how future VLMs should think about vision–language fusion.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on an exceptionally broad suite of public benchmarks spanning multiple capability dimensions (detailed in Appendix A): multimodal reasoning (MMMU, MMMU-Pro, MathVista, MathVision, We-Math, MathVerse, DynaMath, Math-VR, LogicVista, VisualPuzzles, VLMsAreBlind, ZeroBench, VisuLogic — 13 benchmarks), general VQA (MMBench-V1.1, RealWorldQA, MMStar, SimpleVQA — 4 benchmarks), alignment and instruction following (HallusionBench, MM-MT-Bench, MIA-Bench — 3 benchmarks), document understanding including OCR (DocVQA, InfoVQA, AI2D, ChartQA, OCRBench, OCRBench_v2, CC-OCR, OmniDocBench, CharXiv, MMLongBench-Doc — 10 benchmarks), 2D/3D grounding and counting (RefCOCO/+/g, ODinW-13, CountBench, ARKitScenes, Hypersim, SUN RGB-D — 6 benchmarks), embodied and spatial understanding (ERQA, VSI-Bench, EmbSpatialBench, RefSpatialBench, RoboSpatialHome — 5 benchmarks), multi-image understanding (BLINK, MUIRBENCH — 2 benchmarks), video understanding (MVBench, Video-MME, MLVU, LVBench, Charades-STA, VideoMMMU, MMVU — 7 benchmarks), perception with tools (V*, HRBench4K, HRBench8K — 3 benchmarks), multimodal coding (Design2Code, ChartMimic, UniSVG — 3 benchmarks), GUI agent (ScreenSpot Pro, OSWorldG, AndroidWorld, OSWorld, WindowsAA — 5 benchmarks), and text-centric tasks across knowledge (MMLU-Pro, MMLU-Redux, GPQA, SuperGPQA), reasoning (AIME-25, HMMT-25, LiveBench), coding (LiveCodeBench v6, CFEval, OJBench), alignment (IFEval, Arena-Hard v2, Creative Writing v3, WritingBench), agent (BFCL-v3, TAU2-Retail, TAU2-Airline, TAU2-Telecom), and multilingual (MultiIF, MMLU-ProX, INCLUDE, PolyMATH) — over 60 benchmarks total. There is no single test set; each benchmark has its own standard split (e.g., MMMU validation set, DocVQA test set, Video-MME test set). The sheer breadth means no single difficulty distribution applies; the paper instead benchmarks across coarse capability categories.

• Base model(s). The Qwen3-VL family includes six variants: four dense models (2B, 4B, 8B, 32B parameters) and two Mixture-of-Experts models (30B-A3B with 30B total and 3B activated per token; 235B-A22B with 235B total and 22B activated per token), all built on Qwen3 LLM backbones (Yang et al., 2025a) with SigLIP-2-based vision encoders (SigLIP2-SO-400M for the main models, SigLIP2-Large 300M for the 2B and 4B variants). The family is chosen to span a wide range of latency-quality trade-offs, from edge-deployable (2B) to datacenter-scale (235B-A22B), making scaling trends observable across more than two orders of magnitude in activated parameters.

• Metrics. The paper reports accuracy (%) for classification-style benchmarks (MMMU, MathVista, MMBench, MMStar, HallusionBench, etc.), using exact-match grading against ground-truth answers; for OCR and parsing benchmarks, metrics vary — OCRBench uses a composite score (maximum 1000), OmniDocBench uses normalized edit distance (range 0–1), RefCOCO uses accuracy at IoU threshold (not explicitly stated but standard), ODinW-13 uses mean Average Precision (mAP) with detection confidence fixed at 1.0, 3D grounding (ARKitScenes, Hypersim, SUN RGB-D) uses mAP@0.15 with IoU threshold 0.15, Charades-STA uses mIoU, and text-centric benchmarks use standard accuracy or pass@1 metrics. For coding benchmarks, Design2Code uses a 0-100 similarity score rated by GPT-4.1, ChartMimic and UniSVG report task-specific scores. For agent benchmarks, success rate (%) is reported (OSWorld, AndroidWorld, WindowsAA). The diversity of metrics reflects the diversity of tasks; no single summary metric is used across all benchmarks.

• Baselines. The paper compares against a wide range of state-of-the-art models, both open and closed, at comparable scales. For flagship comparisons (Table 2): Gemini 2.5 Pro (Comanici et al., 2025), GPT-5 (OpenAI, 2025), and Claude Opus 4.1 (Anthropic, 2025), each evaluated in both thinking and non-thinking modes where available. For medium-sized models (Table 3): Gemini 2.5 Flash, GPT-5 mini, and the prior-generation Qwen2.5-VL-72B (Bai et al., 2025) as an internal predecessor baseline. For small models (Table 4): GPT-5 Nano. For text-centric tasks (Tables 5–10): Qwen3-235B-A22B-Instruct-2507 and Qwen3-235B-A22B-Thinking-2507 (the text-only counterparts), DeepSeek V3 0324, Claude Opus 4.1, OpenAI o3 (medium), and for smaller scales, Qwen3-32B, Qwen3-30B-A3B, Qwen3-30B-A3B-2507, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, and Qwen3-4B-2507. The prior Qwen2.5-VL family serves as the most important internal baseline, demonstrating generational improvement. Additionally, for the 2D grounding comparison (Section 5.5), conventional open-set object detection specialist models (not named individually) are evaluated, with dataset categories provided simultaneously within the prompt for fair comparison. For video understanding (Section 5.9), the comparison is explicitly flagged as not fully fair due to API limitations: Gemini 2.5 Pro was evaluated with 512 input frames, GPT-5 with 256 frames, and Claude Opus 4.1 with 100 frames, while Qwen3-VL used up to 2,048 frames per video with a cap of 224K video tokens (768 tokens per frame for VideoMMMU and MMVU, 640 for all other benchmarks, sampled at 4 fps for Charades-STA and 2 fps otherwise).

• Generation budget / compute accounting. The paper does not measure test-time compute in generations (as a search-vs-best-of-N paper would) because Qwen3-VL is evaluated in single-pass inference mode: each benchmark example receives one model forward pass (or one chain-of-thought trajectory for thinking models). The primary compute distinction is between thinking and non-thinking (instruct) modes, where thinking models are allowed extended chain-of-thought reasoning with a maximum output length of 32,768 tokens (extended to 81,920 for AIME-25, HMMT-25, and LiveCodeBench v6 to provide "sufficient thinking space"). For the tool-augmented perception benchmarks (V*, HRBench4K, HRBench8K), models are allowed iterative tool calls (image zoom-in) within a structured think-act-analyze loop, making the effective compute per query variable and task-dependent. Sampling hyperparameters are configured separately for instruct models (temperature = 0.7, top-p = 0.8, top-k = 20, presence penalty = 1.5 for the 235B-A22B/32B/30B-A3B; temperature = 1.0, top-p = 1.0, top-k = 40, presence penalty = 2.0 for 8B/4B/2B) and thinking models (temperature = 0.6, top-p = 0.95, top-k = 20 for MoE; temperature = 1.0, top-p = 0.95, top-k = 20 with presence penalty = 1.5 for dense). These are standard sampling settings, not budget-swept, so there is no compute-optimal scaling analysis of the kind seen in inference-time scaling papers — the emphasis is on absolute performance at a fixed, practical inference configuration.

• Cross-validation / statistical protocol. The paper does not report confidence intervals, standard deviations, or statistical significance tests for any benchmark results. No cross-validation is mentioned in the evaluation section. Results are single-point estimates from standard benchmark evaluation protocols. For benchmarks with public test sets, standard evaluation scripts are used. For the self-built multilingual OCR test set (Figure 2), the construction methodology is described (in-house dataset, accuracy threshold of 70% for practical usability) but no statistical protocol is reported. The needle-in-a-haystack experiment (Figure 3) reports per-cell accuracy percentages across a heatmap of video durations and needle depths, but without error bars. This is a limitation: given that some benchmarks have small test sets (e.g., MMMU validation is 900 questions, ZeroBench is 14 questions, MMLongBench-Doc is a few hundred questions), the observed differences between top models (often 1–3 percentage points) may fall within sampling error. The consistency of Qwen3-VL's advantage across dozens of benchmarks provides informal robustness, but the lack of formal statistical treatment is a genuine weakness.

Main Quantitative Results

Multimodal Reasoning (Section 5.2, Tables 2–4)

The flagship Qwen3-VL-235B-A22B-Thinking achieves state-of-the-art results on the core reasoning benchmarks, most notably MathVistamini at 85.8 (Table 2), surpassing Gemini 2.5 Pro's thinking-budget-128 result of 82.7 and GPT-5 high-thinking of 81.3, and substantially ahead of Claude Opus 4.1's non-thinking 74.5. On MathVision (full test set), the thinking variant reaches 74.6, ahead of Gemini 2.5 Pro (73.3 with budget-128) and GPT-5 (70.9 high-thinking). On LogicVista, Qwen3-VL-235B-A22B-Thinking achieves 72.2, the highest reported score, compared to Gemini 2.5 Pro at 72.0 (thinking budget-128) and GPT-5 at 71.8 (high-thinking). The non-thinking variant, Qwen3-VL-235B-A22B-Instruct, establishes itself as the best non-thinking or low-thinking-budget model on MathVistamini (75.5), MathVision (64.3), MathVersemini (70.6), DynaMath (75.1), ZeroBench (3), VLMsAreBlind (77.8), VisuLogic (27.9), and VisualPuzzlesDirect (48.8) (Table 2). The gap between thinking and non-thinking modes varies dramatically by benchmark: on MathVistamini, the thinking variant gains +10.3 points (85.8 vs. 75.5); on MMMU, +2.9 points (80.6 vs. 77.7 for comparable thinking-budget-128 vs. instruct); on HallusionBench, +3.5 points (66.7 vs. 63.2). This variability suggests that extended reasoning helps most on tasks requiring multi-step mathematical derivation or visual-logical deduction, while helping less on knowledge-recall-heavy tasks like MMMU.

The medium-sized Qwen3-VL-32B-Thinking (Table 3) demonstrates a remarkable result: it surpasses the previous-generation flagship Qwen2.5-VL-72B (a model more than 2× its parameter count) on reasoning tasks. On MathVistamini, Qwen3-VL-32B-Thinking achieves 81.9 vs. Qwen2.5-VL-72B-Instruct at 75.3 (the paper's Table 3 specifically compares against Qwen2.5-VL-72B on rows listed in Appendix A, but the 72B results are not directly in Table 3 — the paper states in Section 5.2: "Compared to the previous-generation Qwen2.5-VL-72B model, the medium-sized Qwen3-VL model has already surpassed it on reasoning tasks.") The 30B-A3B MoE variant shows competitive but slightly lower performance than the 32B dense model, with Qwen3-VL-30B-A3B-Thinking achieving 76.0 on MMMU vs. 76.3 for 32B-Instruct (Table 3), confirming that MoE efficiency (activating only 3B of 30B parameters) comes with a modest accuracy trade-off.

Among small models (Table 4), the scaling trend from 2B to 8B is consistent: on MathVistamini, the thinking variants improve from 73.6 (2B) to 79.5 (4B) to 81.4 (8B); on MathVision, from 45.9 to 60.0 to 62.7; on MMMU, from 61.4 to 70.8 to 74.1. The 8B-Thinking model (81.4 MathVistamini, 74.1 MMMU) outperforms GPT-5 Nano's high-thinking mode (71.5 MathVistamini, 75.8 MMMU) on MathVistamini but trails on MMMU, suggesting that the reasoning advantage varies by task type even at small scales.

General Visual Question Answering (Section 5.1, Tables 2–4)

On general VQA, the pattern is consistent across model sizes: Qwen3-VL achieves top-tier or near-top-tier performance. Qwen3-VL-235B-A22B-Instruct achieves 89.3 on MMBench-EN and 88.9 on MMBench-CN (Table 2), the highest non-thinking scores across all compared models (Gemini 2.5 Pro non-thinking: 88.4/86.4; GPT-5 minimal: 79.4/84.9; Claude Opus 4.1: 83.0/74.3). On RealWorldQA, Qwen3-VL-235B-A22B-Instruct reaches 79.2, ahead of Gemini 2.5 Pro (76.0) and GPT-5 (69.9). The thinking variant on RealWorldQA reaches 81.3, though Gemini 2.5 Pro thinking-budget-128 achieves 78.0 and GPT-5 high-thinking achieves 82.8 — the only benchmark where GPT-5's high-thinking mode leads Qwen3-VL-Thinking by a meaningful margin (+1.5 points). On MMStar, Qwen3-VL-235B-A22B-Thinking at 78.7 is second to Gemini 2.5 Pro thinking-budget-128 at 78.5 (the numbers are reversed in Table 2: Qwen3-VL-Thinking 78.7, Gemini 78.5, GPT-5 high-thinking 76.4, Claude Opus 4.1 71.0), making this effectively a statistical tie with Gemini.

At medium scale (Table 3), Qwen3-VL-32B-Thinking scores 89.5 on MMBench-EN and 89.4 on MMBench-CN — slightly higher than the 235B-Instruct variant (89.3/88.9), which is a striking result suggesting that general VQA saturates at moderate model scales and benefits more from thinking-mode reasoning than from raw parameter count. The 30B-A3B MoE variant trails the 32B dense on MMBench (87.0 EN / 85.9 CN for thinking) but remains competitive.

At small scale (Table 4), the 8B-Thinking model achieves 85.3 MMBench-EN / 85.5 MMBench-CN, substantially ahead of GPT-5 Nano (78.4 EN / 77.6 CN for high-thinking). Even the 2B-Thinking model achieves 79.9 EN / 78.8 CN, comparable to GPT-5 Nano's high-thinking, demonstrating that Qwen3-VL's small models are genuinely competitive with similarly-sized alternatives.

Alignment and Subjective Tasks (Section 5.3, Table 2)

On HallusionBench (hallucination detection), Qwen3-VL-235B-A22B-Thinking achieves 66.7, surpassing Gemini 2.5 Pro (63.7), GPT-5 (65.7), and Claude Opus 4.1 (60.4) by margins of 3.0, 1.0, and 6.3 points respectively. This is particularly notable because hallucination benchmarks test a model's ability to resist visual illusions and avoid generating plausible-sounding but incorrect descriptions — a capability that thinking-mode reasoning might be expected to help with, since extended deliberation could allow the model to cross-check its visual interpretation against internal consistency constraints.

On MIA-Bench (complex instruction following), Qwen3-VL-235B-A22B-Thinking achieves 92.7, the highest score across all models (GPT-5 high-thinking: 92.4; Claude Opus 4.1: 90.0; Gemini 2.5 Pro minimal: 91.2). The paper reports a subtask breakdown: Qwen3-VL overtakes GPT-5-high-thinking by 10.0 points in the math subtask and 5.0 points in the textual subtask of MIA-Bench, indicating that the instruction-following advantage is concentrated in domains requiring precise constraint satisfaction. On MM-MT-Bench (multi-turn multimodal instruction following), scores are tightly clustered: Qwen3-VL-235B-A22B achieves 8.5 (both thinking and instruct), Gemini 2.5 Pro reaches 8.4, GPT-5 reaches 7.6, Claude Opus 4.1 reaches 7.9. The narrow range (7.6–8.5 on what is presumably a 1–10 scale, though the paper doesn't specify the maximum) suggests this benchmark may be approaching saturation for top-tier models.

Document Understanding and OCR (Section 5.4, Tables 2–4, Figure 2)

The document understanding results reveal an interesting pattern: the Instruct variant often matches or exceeds the Thinking variant on OCR-heavy tasks. On OCRBench, Qwen3-VL-235B-A22B-Instruct achieves 920 vs. 875 for Thinking (Table 2) — a substantial 45-point gap favoring the non-thinking model. On DocVQA, Instruct achieves 97.1 vs. 96.5 for Thinking. On CC-OCR, Instruct achieves 82.2 vs. 81.5 for Thinking. This suggests that extended chain-of-thought reasoning does not benefit — and may slightly hinder — tasks that are primarily about accurate text extraction rather than multi-step reasoning. The thinking mode's advantage reappears on tasks requiring reasoning over extracted text: on CharXiv (RQ, reasoning subset), Thinking achieves 66.1 vs. 62.1 for Instruct; on MMLongBench-Doc, Instruct achieves 57.0 vs. 56.2 for Thinking (a negligible difference). The flagship model establishes state-of-the-art results on OCRBench (920, the highest among all compared models) and MMLongBench-Doc (57.0 Instruct / 56.2 Thinking, both substantially ahead of GPT-5 at 41.9–44.2 and Claude Opus 4.1 at 43.9–54.5).

Multilingual OCR is a standout capability: Figure 2 shows the model achieving over 70% accuracy on 32 out of 39 tested languages, a major expansion from Qwen2.5-VL's 10 non-English/Chinese languages. The paper doesn't provide a direct comparison point for this multilingual OCR capability against other models (no baseline numbers are reported in Figure 2), but the absolute threshold of 70% is presented as "practical for real-world usability." The training data expansion described in Section 3.2.3 — 30 million in-house OCR samples with pseudo-label refinement, 29 additional languages beyond Qwen2.5-VL's 10, and 30 million synthesized multilingual OCR samples — provides the data foundation for this capability.

At medium scale (Table 3), Qwen3-VL-32B-Instruct achieves 895 on OCRBench and 96.9 on DocVQA, outperforming Gemini 2.5 Flash (853 OCRBench, 92.8 DocVQA) and GPT-5 mini (821/864 OCRBench, 90.5 DocVQA). At small scale (Table 4), Qwen3-VL-8B-Instruct achieves 896 on OCRBench and 96.1 on DocVQA, essentially matching the medium-sized Gemini 2.5 Flash on OCR and significantly outperforming GPT-5 Nano (701 OCRBench, 78.3 DocVQA). This suggests that OCR capability is a particular strength of the Qwen3-VL training recipe, scaling efficiently even to small model sizes.

2D and 3D Grounding (Section 5.5, Tables 2–4)

On RefCOCO (referring expression comprehension, averaged across RefCOCO/+/g), Qwen3-VL-235B-A22B achieves 92.1 (Thinking) / 91.9 (Instruct) (Table 2). The paper notes this is state-of-the-art, though the comparison is limited: Gemini 2.5 Pro reports 74.6 on what appears to be a different evaluation protocol (footnoted with ∗, indicating sourced from the technical report), and Claude Opus 4.1 and GPT-5 do not report RefCOCO results. The ODinW-13 benchmark (open-vocabulary object detection across 13 diverse datasets) provides a more informative comparison: Qwen3-VL-235B-A22B-Instruct achieves 48.6 mAP, while Thinking achieves 43.2. The Instruct advantage here (+5.4 mAP) is substantial, suggesting that for multi-target detection tasks requiring precise localization, the thinking mode's extended reasoning may interfere with or over-complicate the direct grounding task.

On CountBench, Thinking achieves 93.7 and Instruct achieves 93.0, both ahead of Gemini 2.5 Pro (91.0) and Claude Opus 4.1 (91.9). The high absolute scores (all top models above 90%) suggest this benchmark may be nearing saturation.

On 3D grounding (Omni3D benchmark, comprising ARKitScenes, Hypersim, SUN RGB-D), Qwen3-VL-235B-A22B demonstrates a clear advantage on the indoor scene dataset SUN RGB-D: Thinking achieves 34.9 vs. Gemini 2.5 Pro's 29.7, a +5.2 point gap. On ARKitScenes, Thinking achieves 53.7 (no Gemini comparison reported). On Hypersim, Thinking achieves 11.0 (no comparison). The low absolute scores on Hypersim (all models in the 11–14 range) suggest 3D grounding from monocular images remains a very challenging task where current models are far from saturation, and the differences between top models may not be statistically reliable given the small absolute values.

At smaller scales (Tables 3–4), the 2B–8B models achieve non-trivial 3D grounding performance (e.g., 2B-Thinking at 47.7 on ARKitScenes, 11.2 on Hypersim, 28.6 on SUN RGB-D), demonstrating that 3D spatial understanding is retained even at edge-deployment scales.

Embodied and Spatial Understanding (Section 5.8, Tables 2–4)

This category evaluates capabilities that bridge perception and physical reasoning — where objects are in space, how they relate to each other, and what actions are possible. Qwen3-VL-235B-A22B achieves strong results: EmbSpatialBench 84.3 (Thinking) / 83.1 (Instruct), RefSpatialBench 69.9 (Thinking) / 65.5 (Instruct), and RoboSpatialHome 73.9 (Thinking) / 69.4 (Instruct) (Table 2). On EmbSpatialBench, the Thinking variant at 84.3 surpasses Gemini 2.5 Pro's thinking-high of 79.1 and GPT-5 high-thinking of 82.9. On RefSpatialBench, the 69.9 Thinking score dramatically exceeds all baselines: Gemini 2.5 Pro at 36.5 (thinking), GPT-5 at 23.8 (high-thinking), Claude Opus 4.1 at 4.6 (estimated from the reported range in Table 2). This enormous gap — Qwen3-VL more than doubling the next-best model's score — suggests that RefSpatialBench measures a capability (spatial referring with reasoning) that Qwen3-VL's training recipe specifically targets through its synthetic spatial understanding data (Section 3.2.5) and fine-grained pointing annotations, while competing models may not have invested comparably in this capability.

On ERQA (embodied reasoning QA) and VSI-Bench (visual spatial intelligence), the results are more mixed. Qwen3-VL-235B-A22B-Thinking achieves 52.5 on ERQA, behind GPT-5 high-thinking at 65.7 but ahead of Claude Opus 4.1 at 34.8. On VSI-Bench, Thinking achieves 60.0, with no Gemini 2.5 Pro or GPT-5 comparison reported (dashes in Table 2). The pattern suggests that Qwen3-VL's spatial understanding is strongest on tasks requiring precise coordinate-based reasoning (RefSpatialBench, RoboSpatialHome) and less dominant on tasks requiring embodied common-sense reasoning about physical interactions (ERQA).

Multi-Image Understanding (Section 5.7, Table 2)

Multi-image understanding tests the model's ability to compare, contrast, and reason across multiple images simultaneously — a capability that stresses the interleaved multimodal architecture. On BLINK (a benchmark for visual correspondence and difference detection), Qwen3-VL-235B-A22B-Instruct achieves 70.7, slightly ahead of Thinking at 67.1 (Table 2). This is one of the few benchmarks where Instruct outperforms Thinking, suggesting that BLINK's tasks (which often involve rapid visual comparison) are better served by direct perception than extended deliberation. Compared to baselines, Qwen3-VL-235B-A22B-Instruct (70.7) edges out Gemini 2.5 Pro non-thinking (70.0) and GPT-5 minimal (62.8), while Gemini 2.5 Pro thinking-budget-128 achieves 70.6 and GPT-5 high-thinking achieves 71.0 — the top three models are within 0.4 points, a statistical tie.

On MUIRBENCH (robust multi-image understanding), Qwen3-VL-235B-A22B-Thinking achieves 80.1, the highest reported score and substantially ahead of Gemini 2.5 Pro thinking-budget-128 at 77.2 and GPT-5 high-thinking at 77.5. The Instruct variant trails at 73.0, indicating that MUIRBENCH's tasks (multi-hop reasoning across images) benefit substantially from thinking-mode reasoning — a +7.1 point gain over Instruct, one of the largest thinking-mode improvements observed across all benchmarks.

Video Understanding (Section 5.9, Table 2)

The video understanding results show Qwen3-VL competing at parity with the best proprietary models while handling substantially more input frames. On Video-MME (without subtitles), Qwen3-VL-235B-A22B-Instruct achieves 79.2, comparable to Gemini 2.5 Pro thinking-budget-128 at 80.6 and GPT-5 high-thinking at 84.7. On MLVU (multi-task long video understanding), Qwen3-VL-235B-A22B-Instruct achieves 84.3 (averaged across MLVU sub-tasks), surpassing Gemini 2.5 Pro thinking-budget-128 at 81.2 and approaching GPT-5 high-thinking at 86.2. On MVBench, Instruct achieves 76.5, ahead of GPT-5 high-thinking at 75.3. On VideoMMMU (multi-discipline professional video understanding), Thinking achieves 80.0, compared to Gemini 2.5 Pro thinking-budget-128 at 79.4 and GPT-5 high-thinking at 84.6. The pattern is consistent: Qwen3-VL is competitive with the best proprietary models, typically within a few percentage points, while processing 4–8× more input frames (2,048 frames for Qwen3-VL vs. 100–512 for the proprietary models, as detailed in Section 5.9). This frame-count advantage is significant for long-video tasks: on LVBench (extreme long video understanding), Qwen3-VL-235B-A22B-Instruct achieves 67.7, compared to Gemini 2.5 Pro at 73.0 (thinking budget-128) and GPT-5 at 69.0 (thinking-high). The gap to Gemini on LVBench despite the frame-count advantage suggests that frame quantity alone doesn't guarantee superior long-video understanding — architectural quality of temporal modeling also matters.

A critical methodological note: Section 5.9 explicitly states that the frame-count comparison is unfair due to API limitations. Gemini 2.5 Pro was evaluated with 512 frames, GPT-5 with 256 frames, Claude Opus 4.1 with 100 frames — all substantially fewer than Qwen3-VL's 2,048-frame cap. The paper does not report results for Qwen3-VL at reduced frame counts matching the baselines, which would be necessary to separate the effect of architectural quality from the effect of more visual input. This is an important limitation: the video understanding comparison is best interpreted as "Qwen3-VL with 2,048 frames is competitive with proprietary models using fewer frames" rather than "Qwen3-VL's video architecture is superior."

Agent Benchmarks (Section 5.10, Tables 2–4)

On GUI grounding tasks, Qwen3-VL-235B-A22B achieves ScreenSpot Pro at 61.8 (Thinking) / 62.0 (Instruct) (Table 2), with no baseline comparisons reported (dashes for all other models). On OSWorldG (GUI grounding subset of OSWorld), Thinking achieves 68.3 and Instruct achieves 66.7, dramatically ahead of Gemini 2.5 Pro at 45.2 (the only baseline reported). This +23.1 point gap is among the largest advantages Qwen3-VL demonstrates on any benchmark and likely reflects the targeted GUI-agent training data described in Section 3.2.9: large-scale, cross-platform data spanning desktop, mobile, and web environments, with multi-step task trajectories and Chain-of-Thought rationales for planning and decision-making.

On the full interactive agent benchmarks, Qwen3-VL-32B achieves 41.0 on OSWorld and 63.7 on AndroidWorld (Table 3), which the paper states "surpasses the current foundation VLMs." The comparison is limited: Claude Opus 4.1 is the only baseline reported for OSWorld (44.4 — higher than Qwen3-VL-32B's 41.0), and no baselines are reported for AndroidWorld. The Qwen3-VL-235B-A22B scores on OSWorld are 38.1 (Thinking) / 31.6 (Instruct) — unexpectedly lower than the 32B variant (41.0 Thinking), which may indicate that OSWorld's tasks are not bottlenecked by the capabilities that scale with model size, or that the 235B model's reasoning strategies are less well-suited to the interactive, step-by-step nature of GUI tasks compared to the 32B model's more focused training.

On WindowsAA (Windows agent benchmark), Qwen3-VL-32B-Thinking achieves 42.9, substantially ahead of the 235B-Thinking at 32.1 and the 30B-A3B-Thinking at 24.2. This reinforces the observation that agent performance does not monotonically improve with scale and may depend on more nuanced factors like training data composition for specific interaction patterns.

Text-Centric Tasks (Section 5.11, Tables 5–10)

The text-centric evaluation is arguably the paper's most important set of results because it directly tests the central claim that Qwen3-VL matches or surpasses its text-only backbone on language benchmarks. Tables 5–6 compare Qwen3-VL-235B-A22B against leading text-only models, while Tables 7–10 do the same for the 32B/30B-A3B and 8B/4B/2B variants.

Flagship Instruct model (Table 5). Qwen3-VL-235B-A22B-Instruct achieves results that are competitive with or better than Qwen3-235B-A22B-Instruct-2507 (the text-only counterpart) across most benchmarks. On knowledge benchmarks: MMLU-Pro 81.8 vs. 83.0 (text-only leads by 1.2), MMLU-Redux 92.2 vs. 93.1 (text-only leads by 0.9), GPQA 74.3 vs. 77.5 (text-only leads by 3.2), SuperGPQA 60.4 vs. 62.6 (text-only leads by 2.2). The text-only model maintains a modest but consistent advantage on pure knowledge recall. However, on reasoning benchmarks, the pattern reverses: AIME-25 74.7 vs. 70.3 (VL leads by 4.4), HMMT-25 57.4 vs. 55.4 (VL leads by 2.0), LiveBench 74.8 vs. 75.4 (text-only leads by 0.6). On coding: LiveCodeBench v6 54.3 vs. 51.8 (VL leads by 2.5). On alignment: IFEval 87.8 vs. 88.7 (text-only leads by 0.9), Arena-Hard v2 77.4 vs. 79.2 (text-only leads by 1.8), Creative Writing v3 86.5 vs. 87.5 (text-only leads by 1.0). Against external baselines, Qwen3-VL-235B-A22B-Instruct outperforms DeepSeek V3 0324 on AIME-25 (74.7 vs. 46.6), HMMT-25 (57.4 vs. 27.5), LiveBench (74.8 vs. 66.9), Arena-Hard v2 (77.4 vs. 45.6), and LiveCodeBench v6 (54.3 vs. 45.2), establishing clear superiority on reasoning and coding. Against Claude Opus 4.1 (without thinking), Qwen3-VL leads on AIME-25 (74.7 vs. 33.9), HMMT-25 (57.4 vs. 15.9), and LiveCodeBench v6 (54.3 vs. 44.6).

Flagship Thinking model (Table 6). Qwen3-VL-235B-A22B-Thinking achieves results comparable to Qwen3-235B-A22B-Thinking-2507, with the text-only model maintaining advantages on MMLU-Pro (83.8 vs. 84.4), GPQA (77.1 vs. 81.1), AIME-25 (89.7 vs 92.3), HMMT-25 (77.4 vs. 83.9), and LiveCodeBench v6 (70.1 vs. 74.1). The gaps are more pronounced than for Instruct models, suggesting that extended thinking mode amplifies the text-only model's advantage on pure-reasoning tasks. Against external baselines, Qwen3-VL-235B-A22B-Thinking surpasses OpenAI o3 (medium) on AIME-25 (89.7 vs. 88.9) and LiveCodeBench v6 (70.1 vs. 58.6), while tying on HMMT-25 (77.4 vs. 77.5) and trailing on MMLU-Pro (83.8 vs. 85.9).

Medium-sized models (Tables 7–8). The most dramatic result is in the 32B comparison. Qwen3-VL-32B-Instruct utterly dominates Qwen3-32B-Instruct (the text-only counterpart) on reasoning: AIME-25 66.2 vs. 20.2 (+46.0 points), HMMT-25 46.1 vs. 10.9 (+35.2), LiveBench 72.2 vs. 31.3 (+40.9). These are not marginal improvements — they represent a 3.3× improvement on AIME-25, suggesting that at the 32B scale, the text-only model's mathematical reasoning capability is substantially underdeveloped compared to the multimodal version. The gap is smaller but still large for the 30B-A3B variant: AIME-25 69.3 vs. 21.6 (+47.7 for Qwen3-VL vs. Qwen3-30B-A3B-Instruct), though the comparison to Qwen3-30B-A3B-Instruct-2507 (a later text-only checkpoint) narrows to 69.3 vs. 61.3. For thinking models (Table 8), the pattern holds: Qwen3-VL-32B-Thinking achieves AIME-25 83.7 vs. Qwen3-32B-Thinking 72.9 (+10.8), HMMT-25 64.6 vs. 51.8 (+12.8), and the 30B-A3B-Thinking achieves AIME-25 83.1 vs. Qwen3-30B-A3B-Thinking 70.9 (+12.2). Compared to the text-only strong baseline Qwen3-30B-A3B-Thinking-2507, Qwen3-VL-30B-A3B-Thinking trails on AIME-25 (83.1 vs. 85.0) and HMMT-25 (67.6 vs. 71.4), indicating that the later text-only checkpoint has closed some of the gap.

Small models (Tables 9–10). The small-model results confirm that the "VL surpasses text-only" pattern extends to edge scales. Qwen3-VL-8B-Instruct achieves AIME-25 45.9 vs. Qwen3-8B-Instruct 20.9 (+25.0); Qwen3-VL-4B-Instruct achieves 46.6 vs. Qwen3-4B-Instruct 19.1 (+27.5). For thinking models, Qwen3-VL-4B-Thinking achieves AIME-25 74.5 vs. Qwen3-4B-Thinking 65.6 (+8.9), though Qwen3-4B-Thinking-2507 (the later text-only checkpoint) reaches 81.3, exceeding the VL variant by 6.8 points. This temporal pattern — VL models outperforming contemporaneous text-only checkpoints but sometimes trailing later text-only checkpoints — suggests that both modalities benefit from continued training and distillation, and the VL advantage is partly a function of training recipe maturity rather than an inherent property of multimodal learning.

A critical interpretive note: the text-only baselines Qwen3-32B, Qwen3-30B-A3B, Qwen3-1.7B, Qwen3-4B, and Qwen3-8B are not the same checkpoints as the LLM backbones used in Qwen3-VL. The VL models were built from Qwen3 backbones and then further trained on multimodal data with square-root reweighting and strong-to-weak distillation. So the comparison is not "same model + vision training" vs. "same model without vision training" but rather "Qwen3-VL after multimodal training" vs. "Qwen3 text-only model after its own training trajectory." The strong-to-weak distillation applied to VL models (Section 4.3) uses teacher models that the text-only baselines may not have had access to, confounding the comparison.

Perception with Tools (Section 5.6, Table 2)

On fine-grained perception benchmarks that allow tool use (image zoom-in), Qwen3-VL-235B-A22B with tools achieves state-of-the-art results: V 93.7, HRBench4K 85.4, HRBench8K 82.4* (Table 2, Instruct+tool configuration). The tool-augmented performance is substantially higher than without tools: on V*, Instruct+tool reaches 93.7 vs. 85.9 without tools (+7.8 points); on HRBench8K, 82.4 vs. 76.6 without tools (+5.8 points). This demonstrates that the "thinking with images" capability (Section 4.5) provides genuine gains beyond what the base model can achieve in a single forward pass, and the gains from tool use consistently outweigh those from scaling model size: the 8B model with tools achieves 90.1 on V* (Table 4), outperforming the 235B model without tools (85.9).

Multimodal Coding (Table 2)

On front-end reconstruction and SVG generation, Qwen3-VL-235B-A22B-Instruct achieves Design2Code 92.0, ChartMimic 80.5, UniSVG 69.8 (Table 2). The Thinking variant scores slightly lower on Design2Code (93.4) but higher on ChartMimic (78.4 vs. 80.5 for Instruct — actually lower, let me re-check: Table 2 shows Thinking: Design2Code 93.4, ChartMimic 78.4, UniSVG 65.8; Instruct: Design2Code 92.0, ChartMimic 80.5, UniSVG 69.8). The Instruct advantage on ChartMimic (+2.1) and UniSVG (+4.0) alongside the Thinking advantage on Design2Code (+1.4) suggests task-specific effects of reasoning mode on code generation — no consistent pattern. Against baselines, Gemini 2.5 Pro thinking-budget-128 achieves Design2Code 92.5, ChartMimic 62.1, UniSVG 71.7; GPT-5 non-thinking reaches 88.5/85.2/73.0. Qwen3-VL is competitive across all three, with particular strength on ChartMimic (80.5 vs. Gemini's 62.1 and GPT-5's 85.2 — actually trailing GPT-5 by 4.7 points on this benchmark).

---

Ablation Studies and Robustness Checks

Vision Encoder (Qwen3-ViT vs. SigLIP-2): Table 11 compares the in-house Qwen3-ViT against the original SigLIP-2. In CLIP-pretraining zero-shot evaluation, Qwen3-ViT achieves comparable performance on standard benchmarks (ImageNet-1K 84.6 vs. 84.2, ImageNet-V2 78.8 vs. 78.6) but substantially higher on OmniBench, the in-house holistic evaluation suite (45.5 vs. 36.9, +8.6 points). When integrated with the same 1.7B Qwen3 LLM and trained for 1.5T tokens, Qwen3-ViT consistently outperforms SigLIP-2 across VLM benchmarks: AI2D 76.2 vs. 74.1 (+2.1), OCRBench 78.7 vs. 77.2 (+1.5), RealWorldQA 66.1 vs. 58.7 (+7.4), InfoVQA 67.0 vs. 65.3 (+1.7), and OmniBench 53.0 vs. 50.1 (+2.9). The OmniBench advantage is the standout — +8.6 in CLIP pretraining and +2.9 in VLM fine-tuning — confirming that Qwen3-ViT's world knowledge integration transfers through to downstream tasks.

DeepStack: Table 12 ablates DeepStack on an internal 15B-A2B LLM pretrained on 200 billion tokens, evaluated directly on validation sets without post-training. The average across 11 benchmarks improves from 74.7 (baseline) to 76.0 (DeepStack), a modest but consistent +1.3 point gain. The largest improvements cluster on fine-grained perception tasks: InfoVQA +2.3 (71.9 → 74.2), AI2D +1.4 (81.8 → 83.2), OCRBench +2.6 (81.0 → 83.6), DocVQA +1.6 (89.5 → 91.1), ChartQA +1.8 (81.5 → 83.3). Semantic understanding tasks show negligible or mixed effects: TVQA -0.1 (80.6 → 80.5), MMBench-EN +0.2 (81.0 → 81.2), MMBench-CN +0.4 (78.1 → 78.5). This specificity confirms DeepStack's hypothesized mechanism: it primarily benefits tasks requiring access to low-level visual detail, not tasks relying on high-level semantic categorization. The ablation is conducted at a smaller scale (15B-A2B, 200B tokens) than the main models, so the magnitude of benefit at the 235B scale is unknown, but the qualitative pattern is likely to hold.

Needle-in-a-Haystack (Long-Context Retrieval): Figure 3 evaluates Qwen3-VL-235B-A22B-Instruct's ability to locate a semantically salient "needle" frame within long videos. The model achieves 100% accuracy on videos up to 30 minutes (256K tokens, within the trained context window) across all needle depth positions (0% through 100% of the video duration). When extrapolated to longer videos via YaRN-based positional extension, accuracy remains at 99.5% for videos up to 120 minutes (1M tokens). This is a strong result demonstrating that the model's long-context attention mechanisms generalize robustly beyond the training distribution. The heatmap (Figure 3) shows no depth-dependent degradation — the model retrieves needles at the beginning, middle, and end of long videos equally well, suggesting that the interleaved MRoPE and text-based timestamp mechanisms provide uniform temporal discriminability across the full context range.

Non-reported ablations that would strengthen the paper:

• Interleaved vs. chunked MRoPE: The paper describes the interleaved MRoPE design and its motivation (Section 2.1) but does not provide a direct ablation comparing interleaved MRoPE against the original chunked MRoPE from Qwen2.5-VL on a controlled video benchmark. The video understanding improvements (Section 5.9) are attributed to the combination of interleaved MRoPE, text timestamps, and training data scaling, making it impossible to isolate the contribution of interleaved MRoPE specifically.
• Text timestamps vs. T-RoPE: No ablation compares text-based timestamps against the time-synchronized MRoPE from Qwen2.5-VL on temporal grounding tasks. The Charades-STA results (Table 2) show mIoU 64.8 (Instruct) with text timestamps, but no Qwen2.5-VL comparison is provided for this benchmark.
• Square-root reweighting: Despite being highlighted as a key optimization technique in the abstract and Section 1, there is no ablation comparing square-root reweighting against uniform per-token loss or against alternative balancing strategies (e.g., fixed-weight mixing, temperature scaling). The paper claims this "boosts multimodal performance without compromising text capabilities" but provides no controlled experiment isolating this variable.
• Number of DeepStack levels: DeepStack uses three ViT levels (early, middle, late) routed to three LLM layers. No ablation varies the number of levels (e.g., 1, 2, 4, 5) or the mapping between ViT depths and LLM layers, leaving open the question of whether three is optimal.
• Thinking vs. non-thinking compute parity: The thinking models are allowed substantially more output tokens (up to 81,920 for AIME and LiveCodeBench) than non-thinking models, but there is no experiment comparing non-thinking models at equivalent total generation budget (e.g., non-thinking with best-of-N sampling to match the thinking model's token budget). This makes it unclear whether the thinking-mode improvement is due to the structure of chain-of-thought reasoning or simply to increased test-time compute.
• Frame-count normalization for video: As noted, the video comparison uses different frame counts for Qwen3-VL (2,048) vs. baselines (100–512). No ablation evaluates Qwen3-VL at reduced frame counts matching the baselines, which would separate architectural quality from input quantity.

Critical Assessment

The central claim that Qwen3-VL achieves superior multimodal performance is unambiguously supported — the sheer breadth and consistency of the benchmark results across 60+ evaluations, with top-2 or top-1 placement on the majority, provides overwhelming evidence. The more specific and interesting claims, however, require a more nuanced assessment.

Claim: Multimodal training does not erode text-only capability and can even improve it. The evidence for this claim is presented in Tables 5–10, where Qwen3-VL models are compared against Qwen3 text-only models of comparable parameter counts. The most dramatic case — Qwen3-VL-32B-Instruct achieving 66.2 on AIME-25 vs. Qwen3-32B-Instruct at 20.2 — supports the claim but requires careful interpretation. The text-only Qwen3-32B was not the checkpoint used as the LLM backbone for Qwen3-VL-32B; the comparison is between models at different points in their respective training trajectories, with the VL model having undergone strong-to-weak distillation (Section 4.3) that the text-only model may not have benefited from equivalently. Additionally, the Qwen3-32B text-only model may have been trained with different data mixtures, different numbers of optimization steps, and different post-training investments than the VL model. The small-model comparisons (Tables 9–10) show the VL advantage is largest for mid-range checkpoints (Qwen3-4B, Qwen3-8B) and narrows or reverses for later text-only checkpoints (Qwen3-4B-2507, which achieves AIME-25 81.3 vs. Qwen3-VL-4B-Thinking at 74.5). This pattern suggests that the text-only models were still improving with continued training, and the VL models may have captured gains from distillation and post-training that were subsequently incorporated into later text-only checkpoints. So the claim that "multimodal training improves text capability" needs the qualification: relative to contemporaneous text-only counterparts trained with comparable post-training investment. It is not demonstrated that multimodal training is causally responsible for better text performance — it could equally be that the training recipe improvements (distillation, RL, data filtering) applied to VL models, if applied to text-only models, would produce similar gains. A controlled experiment where the identical post-training pipeline is applied to both a text-only and a multimodal model would be needed to establish causality.

Claim: Interleaved MRoPE, DeepStack, and text-based timestamps are responsible for the performance gains. The evidence for architectural contributions comes primarily from the DeepStack ablation (Table 12), which shows a modest +1.3 average gain on an internal 15B-A2B model. The interleaved MRoPE and text-based timestamp mechanisms have no direct ablations reported. The video understanding improvements over Qwen2.5-VL (Table 3 vs. the Qwen2.5-VL-72B baseline) could be attributed to any combination of: architectural improvements, larger training data (the paper describes substantially expanded video data in Section 3.2.7), longer context training (256K vs. Qwen2.5-VL's context window, which the paper doesn't explicitly state but was likely 32K–128K), and the post-training pipeline. Without isolating each architectural change, the contribution of individual components remains unclear. The paper's strength is the integrated system; its weakness is the difficulty of attributing gains to specific design choices.

Claim: Qwen3-VL achieves state-of-the-art performance across multimodal benchmarks. Supported with extensive evidence in Tables 2–4, with the qualification that "state-of-the-art" depends on the specific benchmark and mode. Qwen3-VL-235B-A22B-Thinking leads on MathVistamini (85.8), MathVision (74.6), LogicVista (72.2), MUIRBENCH (80.1), HallusionBench (66.7), and MIA-Bench (92.7), among others. It ties or trails within a few points on MMMU (80.6 vs. Gemini's 81.7), MMMU-Pro (69.3 vs. GPT-5's 71.2), MathVerse (85.0 vs. GPT-5's 84.1), and Video-MME (79.0 vs. GPT-5's 84.7). This is a genuinely competitive position — no single model dominates all benchmarks. The appropriate characterization is that Qwen3-VL is "among the top tier" across the board and "best-in-class" on a meaningful subset of reasoning and document understanding benchmarks. The missing comparison that would strengthen this assessment is a FLOPs-matched or training-compute-matched comparison against proprietary models — without knowing the training budgets of Gemini 2.5 Pro or GPT-5, it's impossible to say whether Qwen3-VL achieves its results more efficiently or simply benefits from comparable scale.

Genuine weaknesses in the experimental design:

• Single model family. All results are from the Qwen3-VL family. There is no demonstration that the architectural innovations (interleaved MRoPE, DeepStack, text timestamps) generalize to other VLM architectures or base LLMs. A study applying these techniques to, say, a LLaMA-based VLM would test generalizability but is absent.

• No compute-matched comparisons across model sizes. The paper demonstrates that Qwen3-VL-8B is competitive with much larger models on some benchmarks (e.g., OCRBench 896 vs. Gemini 2.5 Flash at 853), but there is no systematic analysis of performance per unit of training compute or inference FLOPs. This makes it difficult to assess whether the scaling is efficient or whether the 235B model's advantages justify its ~30× inference cost over the 8B model.

• Benchmark saturation and ceiling effects. Several benchmarks show top models clustered within 1–3 points (MMBench, MMStar, MM-MT-Bench), suggesting saturation where differences reflect noise rather than genuine capability gaps. The paper does not report confidence intervals or statistical tests, so the reliability of fine-grained rankings on near-saturated benchmarks is unknown.

• Omitted evaluations for proprietary baselines. Many cells in Tables 2–4 contain dashes for proprietary models, indicating benchmarks where the comparison could not be made due to lack of public results or API access. This is unavoidable when comparing against closed-source models but means some claims of leadership (e.g., on ODinW-13, ARKitScenes, EmbSpatialBench) are based on incomplete comparisons.

• The "multimodal improves text" claim conflates training recipe and modality. The strong-to-weak distillation pipeline (Section 4.3) is described as distilling from teacher models using text-only data to fine-tune the LLM backbone. This is fundamentally a text-only training step applied to the VL model, not a benefit of multimodality. If this distillation were applied to the text-only Qwen3 models, would they achieve similar gains? The paper cannot answer this because it didn't run the experiment.

Missing experiments that would have strengthened the paper:

1. Ablation of interleaved MRoPE vs. chunked MRoPE on a controlled video benchmark to isolate the contribution of balanced frequency spectrum.
2. Ablation of text timestamps vs. T-RoPE on temporal grounding tasks (Charades-STA) to measure the specific benefit of explicit time encoding.
3. Square-root reweighting ablation against uniform loss weighting on a held-out set of text and multimodal benchmarks to validate the balancing claim.
4. Frame-count normalized video comparison — evaluate Qwen3-VL at 256 or 512 frames to match proprietary baselines and isolate architectural quality from input quantity.
5. Text-only distillation control — apply the same strong-to-weak distillation pipeline to text-only Qwen3 models and compare against Qwen3-VL to determine whether distillation, not multimodality, drives the text performance gains.
6. Statistical significance testing on benchmarks with small test sets (ZeroBench: 14 questions, MMLongBench-Doc: a few hundred) where 1–2 point differences are likely within sampling error.

6. Limitations and Trade-offs

6.1 No Causal Evidence That Multimodal Training Improves Text Capability

The assumption: The paper claims as a central contribution that "multimodal models are expected to match or surpass their text-only counterparts on language benchmarks" (Section 1), and the evaluation in Tables 5–10 shows Qwen3-VL models outperforming Qwen3 text-only models on many text-centric benchmarks. The implied causal claim is that adding visual training, properly done, enhances linguistic reasoning rather than degrading it. However, the comparison between Qwen3-VL and Qwen3 text-only models is not a controlled experiment. The VL models underwent a post-training pipeline — including strong-to-weak distillation (Section 4.3) and reinforcement learning (Section 4.4) — that the text-only baselines may not have received equivalently. The paper states that distillation was performed "using text-only data to fine-tune the LLM backbone" (Section 4.1), which is fundamentally a text-only training step, not a benefit of multimodality itself.

The consequence: The headline result — Qwen3-VL-32B-Instruct scoring 66.2 on AIME-25 versus Qwen3-32B-Instruct at 20.2 (Table 7) — cannot be attributed specifically to multimodal training. The gain could arise entirely from the distillation and RL pipeline applied to the VL model, which the text-only counterpart may not have received with comparable investment. If this is the case, the paper's framing — that multimodal training is a net positive for text capability rather than a trade-off — would be misleading. The distinction matters practically: an organization deciding whether to invest in multimodal training versus enhanced text-only post-training needs to know which intervention actually drives the text improvements. The paper provides no basis for deciding. Furthermore, where later text-only checkpoints are available (e.g., Qwen3-4B-Thinking-2507 at 81.3 on AIME-25 vs. Qwen3-VL-4B-Thinking at 74.5, Table 10), the text-only model surpasses the VL variant, suggesting that the gap may be an artifact of comparing models at different points in their training maturity rather than a stable property of multimodal training.

What evidence exists: The text-centric evaluation tables (5–10) show the comparison but do not control for post-training investment. The paper does not report how many GPU-hours or training tokens were spent on post-training for Qwen3-VL versus Qwen3 text-only models. There is no ablation where an identical distillation-and-RL pipeline is applied to a text-only Qwen3 backbone and compared against the VL variant.

Mitigation status: The paper does not acknowledge this as a limitation. The strong-to-weak distillation section (4.3) is presented as part of the Qwen3-VL training recipe, not as a confound for the text-capability claim. No control experiment is suggested for future work.

---

6.2 Architectural Contributions Are Not Isolated Through Ablations

The assumption: The paper attributes performance gains to three architectural innovations — interleaved MRoPE (Section 2.1), DeepStack (Section 2.2), and text-based video timestamps (Section 2.3) — and presents these as key technical contributions in both the abstract and the model architecture section. The implicit claim is that each innovation provides a measurable, independent improvement over the prior Qwen2.5-VL design.

The consequence: Only DeepStack receives a direct ablation (Table 12, on a 15B-A2B model at 200B tokens). Interleaved MRoPE and text-based timestamps have no controlled comparison against their Qwen2.5-VL counterparts (chunked MRoPE and T-RoPE, respectively). The video understanding improvements over Qwen2.5-VL-72B (Section 5.9) could be driven by any combination of: the architectural changes, the substantially expanded video pretraining data (Section 3.2.7 describes a "temporal-aware video understanding" pipeline with dense caption synthesis and spatio-temporal grounding that did not exist in Qwen2.5-VL), the 256K context window (vs. a smaller window for Qwen2.5-VL, though the paper does not explicitly state Qwen2.5-VL's context length), the added post-training stages, or simply scaling from 72B to 235B parameters. A practitioner trying to improve their own VLM cannot determine which of these innovations is worth implementing, because the paper does not separate their effects. This reduces the paper's value as a source of generalizable design principles.

What evidence exists: The DeepStack ablation (Table 12) shows a +1.3 average gain across 11 benchmarks, with larger improvements on fine-grained perception tasks. This is the only architectural ablation in the paper. For interleaved MRoPE, the paper provides a conceptual motivation (balanced frequency spectrum) but no empirical isolation. For text timestamps, the paper contrasts the approach with T-RoPE conceptually (Section 2.3) but does not report, for example, Charades-STA performance with T-RoPE vs. text timestamps while holding all else constant.

Mitigation status: The paper does not present the lack of architectural ablations as a limitation. The innovations are described as an integrated package, and the DeepStack ablation demonstrates willingness to isolate components, making the absence of MRoPE and timestamp ablations a notable gap rather than an acknowledged constraint.

---

6.3 Video Understanding Comparison Is Confounded by Unequal Input Frame Counts

The assumption: The paper compares Qwen3-VL's video understanding performance against Gemini 2.5 Pro, GPT-5, and Claude Opus 4.1 (Section 5.9, Table 2) and draws conclusions about competitive or superior performance. However, the evaluation protocol uses substantially different numbers of input frames across models. Section 5.9 explicitly states: "our comparison cannot guarantee full fairness due to resource and API limitations, which constrained the number of input frames used during evaluation: 512 for Gemini 2.5 Pro, 256 for GPT-5, and 100 for Claude Opus 4.1." Qwen3-VL was evaluated with up to 2,048 frames per video (4–20× more than baselines), with a cap of 224K video tokens.

The consequence: The video benchmark results are uninterpretable as comparisons of architectural quality. Qwen3-VL's competitive performance on Video-MME (79.0 Thinking vs. Gemini's 80.6 with thinking-budget-128, Table 2) could mean that Qwen3-VL's video architecture is genuinely strong, or that its architectural quality is meaningfully worse but compensated for by having 4× more visual information, or any combination in between. The 2,048-frame evaluation represents a different inference-compute budget — processing more frames costs proportionally more FLOPs and memory. Without a frame-count-normalized comparison, a practitioner cannot determine whether Qwen3-VL's video processing is more efficient, equally efficient, or less efficient than competing models per unit of visual input. This is particularly important for deployment scenarios where frame count is constrained by latency or memory (e.g., real-time video understanding, on-device processing), since Qwen3-VL's apparent advantage may require compute budgets that are unavailable in those settings.

What evidence exists: Section 5.9 documents the frame-count discrepancy explicitly and transparently, which is commendable. However, the paper provides no ablation evaluating Qwen3-VL at 256 or 512 frames to match the proprietary baselines, nor does it report Qwen3-VL's performance as a function of frame count. There is no evidence in the paper that would allow a reader to estimate Qwen3-VL's performance at reduced frame budgets.

Mitigation status: The paper acknowledges the unfairness ("our comparison cannot guarantee full fairness") but treats it as a logistical constraint rather than a limitation of the claims. The video benchmark results in Table 2 are presented without the frame-count caveat in the caption or table notes. The acknowledgment is buried in the prose of Section 5.9, making it easy for a reader scanning the results tables to miss the confound.

---

6.4 Thinking-Mode Performance Gains Are Not Separated from Test-Time Compute Effects

The assumption: The paper presents both "thinking" and "non-thinking" (instruct) variants of Qwen3-VL and attributes the thinking variant's superior performance on reasoning benchmarks to its Chain-of-Thought reasoning capability (Section 4.2.2, "Long-CoT Cold Start Data"). Thinking models are trained on reasoning trajectories that explicitly model intermediate steps, and at inference time they are allowed extended output lengths — up to 32,768 tokens for most benchmarks and 81,920 tokens for AIME-25, HMMT-25, and LiveCodeBench v6 (Section 5.11).

The consequence: The thinking variant's advantage could arise from two fundamentally different sources: (1) the structure of chain-of-thought reasoning (breaking problems into steps, self-verification, exploration of alternatives), which would be a genuine capability improvement, or (2) the quantity of test-time compute consumed (generating more output tokens before producing a final answer), which is a compute-scaling effect rather than a reasoning-quality effect. The paper provides no experiment that distinguishes these explanations. Without a comparison where the non-thinking model is allowed an equivalent total generation budget — for example, via best-of-N sampling with N chosen to match the thinking model's average output token count — it is impossible to determine how much of the thinking-mode improvement is due to reasoning structure versus brute-force compute. A practitioner deciding whether to deploy thinking or non-thinking mode for a latency-sensitive application needs to know: if I give the non-thinking model 10× more output tokens through parallel sampling, does it match the thinking model? The paper provides no answer.

What evidence exists: The thinking-vs-non-thinking gaps vary dramatically by benchmark. On MathVistamini, the 235B-Thinking model leads 235B-Instruct by +10.3 points (85.8 vs. 75.5, Table 2). On MMLU, the gap is +2.9 points (80.6 vs. 77.7 for comparable configurations). On HallusionBench, +3.5 points (66.7 vs. 63.2). The variability suggests that extended reasoning helps more on some tasks than others, but without a compute-matched baseline, the source of task-specific variation is unclear. The paper also reports that on OCR-heavy tasks like OCRBench and DocVQA, the Instruct variant sometimes outperforms Thinking (e.g., OCRBench 920 vs. 875, Table 2), indicating that extended reasoning can actively harm performance on certain task types. This negative effect is noted as an observation but not explored systematically.

Mitigation status: The paper does not acknowledge the test-time compute confound. The thinking variants are presented as a capability distinction (models that reason vs. models that don't), not as a compute-allocation distinction (models that spend more inference FLOPs vs. models that spend fewer). No compute-normalized comparison is suggested as future work.

---

6.5 Difficulty Estimation for Real-World Deployment Is Not Addressed

The assumption: The paper presents Qwen3-VL as "a foundational engine for image-grounded reasoning, agentic decision-making, and multimodal code intelligence in real-world workflows" (Abstract) and evaluates it on a broad suite of standardized benchmarks. The implicit assumption is that benchmark performance predicts real-world deployment performance. However, the benchmarks used are curated, static test sets with known answer formats, clean images, and well-defined tasks. Real-world deployment involves highly variable input quality (blurry images, unusual lighting, rotated text, noisy video), adversarial or edge-case queries, long-tailed visual concepts, and tasks where the "correct" answer is ambiguous or multi-faceted. The paper provides almost no evidence about Qwen3-VL's robustness to distribution shift or input degradation.

The consequence: A practitioner deploying Qwen3-VL in a production setting — for example, a document processing pipeline that receives smartphone photos of paperwork, or a video monitoring system with low-resolution feeds — has no information about how performance degrades under realistic input conditions. Benchmarks like MMBench, MMStar, and DocVQA use relatively clean, well-framed images. Real-world OCR on crumpled receipts, rotated signage, or low-contrast text may yield substantially different results, and the paper provides no basis for estimating the degradation. The multilingual OCR evaluation (Figure 2) is the only test on an in-house dataset, and it reports only accuracy for 39 languages on what is presumably clean, well-lit text — there is no analysis of performance as a function of image quality, resolution, rotation, or partial occlusion.

What evidence exists: The paper includes some benchmarks that test robustness indirectly: HallusionBench tests resistance to visual illusions, RealWorldQA tests spatial understanding on real-world photographs, and ODinW-13 tests open-vocabulary detection across diverse domains. Results on these benchmarks are strong — HallusionBench 66.7 (Table 2), RealWorldQA 79.2, ODinW-13 48.6 mAP — which provides partial evidence of generalization. However, these are still standardized benchmarks with controlled construction, not uncontrolled real-world streams. The paper provides no analysis of failure modes, per-category performance breakdowns, or confidence calibration (does the model know when it's uncertain on out-of-distribution inputs?).

Mitigation status: The paper does not acknowledge distribution-shift robustness as a limitation or evaluation gap. There is no discussion of known failure modes in the conclusion (Section 6). This is standard for technical reports of this type but represents a genuine practical risk for downstream deployment.

---

6.6 The Open-Source Release Does Not Include Training Data or Full Training Reproducibility

The assumption: The paper states that Qwen3-VL is released "under the Apache 2.0 license" (Section 6) with the goal of catalyzing "community-driven innovation." The implicit promise of open-weight release is that the research community can build upon, fine-tune, and adapt the model for novel applications. However, the paper does not release the training data, the data construction pipelines, or the detailed training configurations (exact data mixtures per stage, hyperparameter schedules, RL reward function details) needed to reproduce or extend the training process.

The consequence: The community can use Qwen3-VL as a fixed-capability model for inference and can fine-tune it on new data, but cannot reproduce the pretraining or post-training process, ablate design choices not covered in the paper, study the effect of data composition on specific capabilities, extend the training to add new modalities or languages using the same methodology, or audit the training data for biases, toxic content, or copyright concerns. Critically, the data construction sections (3.2.1–3.2.9) describe data sources and pipelines in general terms — "a large-scale corpus of contemporary, predominantly Chinese–English multilingual image–text pairs from web sources" (Section 3.2.1), "30 million in-house collected samples" (Section 3.2.3), "over 60 million K–12 and undergraduate-level exercises" (Section 3.2.8) — but do not provide the actual datasets, the filtering code, or even precise data volume breakdowns per source. This means the paper's claims about the importance of data quality, diversity, and specific data mixtures cannot be independently verified or replicated.

What evidence exists: The paper is transparent about its training stages (Table 1 provides token budgets and sequence lengths for each stage) and provides a high-level description of data sources for each category. It reports results on public benchmarks using standard evaluation protocols, with prompts provided in Appendix B for reproducibility of evaluation. However, no training data is released, no data card with detailed composition statistics is provided, and the filtering pipeline descriptions (Section 4.2.1) are algorithmic but not accompanied by the trained filtering models or filtering thresholds.

Mitigation status: The paper does not explicitly describe this as a limitation — it is standard practice for large-scale model releases to withhold training data for legal, competitive, or safety reasons. The Apache 2.0 license applies to model weights, not data. However, for a paper claiming to provide a foundation for community innovation, the inability to reproduce or extend the training process represents a genuine constraint on the "foundational engine" vision, since downstream adaptations that require retraining (new languages, new modalities, domain-specific pretraining) cannot follow the paper's recipe.

7. Implications and Future Directions

How This Work Changes the Landscape

Qwen3-VL changes the landscape by demonstrating—at scale and across six model sizes—that multimodal training need not erode text-only capability. This is not an incremental improvement but a reframing of the core assumption that has governed VLM development: that adding vision necessarily trades off against language. The paper's evidence in Tables 5–10, where Qwen3-VL models match or outperform their text-only Qwen3 counterparts on reasoning benchmarks (AIME-25: Qwen3-VL-32B-Instruct at 66.2 vs. Qwen3-32B-Instruct at 20.2, a 3.3× improvement), shifts the burden of proof. The question is no longer "how do we minimize language erosion during multimodal training?" but rather "what training recipes enable vision to enhance language reasoning, and why did prior recipes fail to achieve this?" The paper credits the combination of square-root reweighting, careful data mixture composition, and strong-to-weak distillation, though—as discussed in Section 6.1—the causal attribution between these mechanisms remains unresolved.

The paper provides a unified architectural framework for multimodal positional encoding through the interleaved MRoPE diagnostic. Prior to Qwen3-VL, the MRoPE design (Qwen2-VL, Qwen2.5-VL) chunked embedding dimensions into separate temporal, horizontal, and vertical subspaces. The Qwen3-VL team identified that this creates an imbalanced frequency spectrum—each axis receives only a fraction of the total frequency bandwidth—and that this imbalance specifically degrades long-video temporal discrimination. The interleaved fix (distributing t, h, and w uniformly across all frequency bands) is a diagnostic contribution as much as a methodological one: it establishes that designers of multimodal positional encodings should check whether each positional axis has uniform coverage of the full frequency range, because imbalance produces axis-specific degradation that may be invisible on short-context evaluations. This principle generalizes beyond MRoPE to any embedding scheme that partitions representational capacity across multiple positional axes.

The paper also shifts the temporal grounding paradigm from implicit to explicit encoding. Qwen2.5-VL's T-RoPE tied temporal position IDs to absolute time through the rotary embedding mechanism—an elegant continuous representation that proved fragile for long videos due to sparse, large position IDs and training data coverage requirements. Qwen3-VL's move to text-based timestamps (e.g., <3.0 seconds>) treats time as content rather than position, leveraging the LLM's existing linguistic knowledge of numbers and temporal expressions. This resolves a contradiction in prior work: continuous temporal encodings were mathematically appealing but empirically limited; explicit symbolic encodings had been considered too costly in context length. Qwen3-VL shows the context-length cost is modest (a few tokens per frame) and the precision gain is substantial, particularly for long-video tasks like temporal grounding (Charades-STA mIoU 64.8, Table 2) and video QA. This finding redirects research on temporal modeling in VLMs: rather than designing increasingly sophisticated continuous time encodings, the more productive path may be to simply tokenize time and let the language model handle it.

The most forward-looking landscape shift comes from the "thinking with images" capability (Section 4.5). Qwen3-VL demonstrates that a VLM can be trained, through multi-turn tool-integrated RL, to actively invoke visual processing tools (image zoom-in, region analysis) during its reasoning process, rather than being a passive consumer of a fixed visual input. The results on V* (93.7 with tools vs. 85.9 without, Table 2) and HRBench8K (82.4 with tools vs. 76.6 without) show that tool use provides gains that exceed those from scaling model size—the 8B model with tools achieves V* 90.1, outperforming the 235B model without tools (85.9). This opens a research direction where VLM capability is understood not as a fixed property of model weights but as a function of the model's ability to strategically acquire and process visual information at inference time. It also provides a concrete architecture for agentic VLMs that must interact with dynamic visual environments (GUIs, robotics, augmented reality) where the relevant visual information is not knowable from a single static image.

Follow-Up Research This Work Enables

Isolating the causal effect of multimodality on text reasoning. The paper shows Qwen3-VL outperforming Qwen3 on text-centric benchmarks (Tables 5–10), but the VL models received strong-to-weak distillation and RL that the text-only baselines may not have received equivalently. A controlled experiment would take an identical Qwen3 checkpoint, apply the identical post-training pipeline (distillation from the same teacher, same RL tasks and rewards, same number of optimization steps) to produce two models—one that sees only text data during post-training, one that sees the multimodal mixture—and evaluate both on text benchmarks. If the multimodal variant still outperforms, this would establish that visual training data causally improves linguistic reasoning, perhaps because visual examples of mathematical concepts or logical structures provide grounding that text alone cannot. If the text-only variant matches or exceeds the multimodal variant, then the paper's headline result is an artifact of post-training investment, not a benefit of multimodality, and the field should redirect effort from multimodal data scaling to post-training recipe optimization for text capability.

Frame-count-normalized video understanding comparison. Section 5.9 acknowledges that Qwen3-VL was evaluated with 2,048 frames per video while Gemini 2.5 Pro used 512, GPT-5 used 256, and Claude Opus 4.1 used 100. A direct follow-up would evaluate Qwen3-VL-235B-A22B on Video-MME, MLVU, and LVBench at 256 and 512 frames (matching the proprietary baselines) and at 100 frames (matching Claude), producing scaling curves of accuracy vs. frame count. These curves would answer: does Qwen3-VL's video architecture extract more understanding per frame than competing models, or does it simply compensate for a weaker architecture with more visual input? The experimental setup is straightforward—subsample the existing evaluation videos to the target frame counts and re-run inference—and the results would determine whether Qwen3-VL's interleaved MRoPE and text-timestamp design genuinely advances temporal modeling, or whether the paper's video results are primarily a story about input quantity.

Ablation of interleaved MRoPE vs. chunked MRoPE on long-video temporal grounding. The paper motivates interleaved MRoPE conceptually (Section 2.1) but provides no controlled comparison against the original chunked MRoPE from Qwen2.5-VL. A clean ablation would train two otherwise-identical Qwen3-VL variants—one with interleaved MRoPE, one with chunked MRoPE—on the same pretraining data, then evaluate on Charades-STA (temporal action localization), YouCook2 (dense video captioning with timestamps), and a needle-in-a-haystack video retrieval task at multiple context lengths (8K, 32K, 128K, 256K). The prediction from the paper's frequency-imbalance hypothesis is that the interleaved variant should show a growing advantage as context length increases, with the largest gap on the longest videos. If the gap is negligible or appears only at extreme context lengths, the frequency-imbalance story would be empirically weaker than the paper suggests, and the video improvements would be more attributable to data scaling or text timestamps.

Dynamic difficulty-aware multimodal inference. The paper demonstrates that thinking-mode reasoning improves performance on complex reasoning tasks (MathVistamini +10.3 points, Table 2) but can hurt or not help on perception-heavy tasks (OCRBench 875 Thinking vs. 920 Instruct, Table 2). This opens the question of whether a single model can dynamically decide how much reasoning to apply per query, rather than deploying a fixed thinking or non-thinking variant. A follow-up would train a lightweight router (possibly a linear classifier on top of the LLM's hidden states after the first few layers of processing) that predicts, for a given input, whether extended chain-of-thought reasoning is likely to improve or degrade the answer. The router could be trained on the per-benchmark thinking-vs-non-thinking performance differences observed in Tables 2–4, with the goal of achieving the best of both modes—thinking accuracy when it helps, non-thinking efficiency when it doesn't. This connects to the broader literature on adaptive computation and inference-time scaling, but applied to the specific architecture of a VLM with two distinct output modes.

Stress-testing DeepStack on extreme fine-grained perception. The DeepStack ablation (Table 12) shows gains of +2.3 on InfoVQA and +1.6 on DocVQA, suggesting the mechanism helps with fine-grained visual detail. A stress-test would evaluate DeepStack on a purpose-built benchmark that systematically varies the spatial scale of task-relevant information: for example, document images where the critical text is rendered at font sizes from 4pt to 24pt, or object counting tasks where target objects occupy 0.1% to 10% of image area. If DeepStack's benefit grows as the relevant features become smaller, this would confirm the mechanism's role in preserving fine-grained information through the vision-language bottleneck. If the benefit is constant across scales, DeepStack may be helping through a different mechanism (e.g., providing richer training signal during pretraining rather than better features at inference). The experiment would also probe whether three DeepStack levels (early, middle, late ViT features) is optimal, or whether adding more levels (e.g., 5 or 6) continues to improve fine-grained perception or saturates.

Extending text-based timestamps to other continuous metadata. The paper's text-based timestamp approach (Section 2.3) treats time as explicit text tokens. The same principle could apply to other continuous metadata that VLMs currently encode implicitly: GPS coordinates for geo-tagged images, depth values for RGB-D inputs, sensor readings for robotics, or even abstract metadata like "confidence" or "importance" scores. A follow-up would take an existing VLM that encodes 3D spatial position through learned embeddings and replace the embedding with text tokens (e.g., <depth 3.2 meters>), then evaluate on 3D grounding tasks (ARKitScenes, Hypersim, SUN RGB-D from Table 2) and spatial reasoning benchmarks. If text-based depth encoding matches or exceeds learned embedding performance, this would establish a general design principle: LLMs already understand numbers and units from text pretraining; explicit symbolic metadata leverages that prior knowledge more efficiently than learning continuous encodings from scratch. The experiment would also measure the context-length cost of explicit metadata tokens at scale, determining the practical limits of this approach.

Practical Applications and Downstream Use Cases

Multilingual document digitization at scale. Qwen3-VL achieves over 70% OCR accuracy on 32 of 39 tested languages (Figure 2) and 920 on OCRBench (Table 2), with strong performance extending to the 2B and 4B edge-scale models (OCRBench 858 and 881 respectively, Table 4). This enables a deployment architecture where a lightweight Qwen3-VL-4B model runs on-device or at the edge for real-time document scanning in any of 39 languages—receipt processing, form digitization, signage translation—with accuracy sufficient for practical use without cloud round-trips. Documents requiring higher accuracy or complex layout parsing (multi-column PDFs, tables, forms) can be routed to the cloud running Qwen3-VL-235B-A22B, which achieves state-of-the-art OmniDocBench and CC-OCR results (Table 2). The 256K context window means multi-page documents up to hundreds of pages can be processed in a single forward pass (MMLongBench-Doc 57.0, Table 2), eliminating the need for chunking and reassembly pipelines that introduce errors at page boundaries. The key deployment number: Qwen3-VL-8B-Instruct achieves 96.1 on DocVQA and 896 on OCRBench (Table 4), performance that would have required a large proprietary model just a year ago, while running on a single GPU at interactive latency.

GUI agent automation with open-weight models. Qwen3-VL-32B achieves 41.0 on OSWorld and 63.7 on AndroidWorld (Table 3), results that the paper states surpass current foundation VLMs. These are interactive benchmarks where the model must perceive a GUI screenshot, decide on an action (click, type, scroll), execute it, and adapt to the resulting screen state—a loop that requires robust visual grounding, instruction following, and error recovery. The Qwen3-VL-32B model can be deployed as an open-weight alternative to proprietary GUI agents for tasks like automated software testing, accessibility assistance (navigating interfaces for users with motor impairments), robotic process automation for legacy software without APIs, and mobile task automation (booking appointments, filling forms, data entry across multiple apps). The 32B size is deployable on a single high-memory GPU or quantized to run on consumer hardware, making this capability accessible to organizations that cannot use cloud-based proprietary models due to data privacy, cost, or latency constraints. The ScreenSpot Pro results (57.1–60.5 for medium models, Table 3) indicate the GUI grounding—correctly identifying which UI element to interact with—works across desktop, mobile, and web interfaces, providing the perception foundation for reliable agent behavior.

Long-video understanding for surveillance, content moderation, and video search. Qwen3-VL's 256K context window with 99.5% needle-in-a-haystack accuracy at 1M tokens (Figure 3) means it can process videos up to 2 hours in a single inference pass while reliably retrieving specific events. The MLVU average of 84.3 (Table 2, Instruct) demonstrates competence across diverse long-video tasks including summarization, question answering, and anomaly detection. This enables a deployment where security footage (typically hours of continuous recording) is processed end-to-end without the sliding-window or keyframe-extraction approaches that dominate current video understanding pipelines and that routinely miss events spanning window boundaries. A monitoring system could ingest a full shift of camera footage, locate all instances of a described event ("person in red jacket entering the warehouse between 2:00 and 3:00"), and provide timestamped clips—all in one model call. The text-based timestamp mechanism (Section 2.3) means the temporal grounding is precise and human-readable, enabling integration with existing video management systems that expect timestamp-indexed results.

STEM education and assessment with visual reasoning. Qwen3-VL-235B-A22B-Thinking achieves 85.8 on MathVistamini and 74.6 on MathVision (Table 2), benchmarks that test mathematical reasoning over diagrams, graphs, and geometric figures. The model's thinking mode produces explicit step-by-step reasoning (trained on 12 million multimodal reasoning samples with validated CoT trajectories, Section 3.2.8), making it suitable for educational applications where the reasoning process matters as much as the final answer. A tutoring system could present a geometry problem with a diagram, have Qwen3-VL produce a chain-of-thought solution, and surface the reasoning steps for student review—including flagging steps where the model is uncertain or where alternative approaches exist. The model's ability to handle both text-only and multimodal math (Tables 5–10 show strong text math as well) means it can assist across the full STEM curriculum, from purely symbolic algebra to diagram-based physics problems. At the 8B scale (MathVistamini 81.4, Table 4), this capability runs on edge devices, enabling offline tutoring applications in low-connectivity environments.

When to Prefer This Method

The paper does not articulate an explicit decision framework positioning Qwen3-VL against named alternative architectures or training approaches. The comparisons in Section 5 are against specific proprietary and open-weight models, not against alternative design philosophies (e.g., "prefer interleaved MRoPE over chunked MRoPE when video tasks constitute >30% of your workload"). The architectural innovations are presented as integrated upgrades to the Qwen VLM lineage, and the training recipe is presented as a holistic pipeline rather than a set of modular choices with characterized trade-offs. Without explicit positioning against alternatives in the paper, a decision matrix would be speculative rather than grounded in the authors' claims.