SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features

ArXiv: 2502.14786

🎯 Pitch

A single training recipe combining decoder-based losses and self-supervised objectives transforms a standard vision-language encoder into a model that simultaneously boosts zero-shot classification and referring expression comprehension by over 18 points. The frozen representations also translate directly into large gains for dense prediction tasks like segmentation, without sacrificing core retrieval performance. This unified approach ships in a new open-weight family, SigLIP 2, which is also natively multilingual and backward-compatible with its predecessor.

---

1. Executive Summary

This paper introduces SigLIP 2, a family of multilingual vision-language encoders that unifies several independently developed techniques—captioning-based pretraining via LocCa (attaching a transformer decoder for auxiliary tasks like grounded captioning and referring expression prediction), self-supervised losses from SILC and TIPS (self-distillation and masked prediction to improve dense feature quality), and active data curation (implicit distillation through ACID-based example selection)—into a single training recipe built on the original SigLIP sigmoid loss. Evaluated across zero-shot classification, image-text retrieval, and transfer as a frozen vision encoder for VLMs on benchmarks including ImageNet, COCO, Flickr, and Crossmodal-3600, SigLIP 2 models at ViT-B, L, So400m, and g scales consistently outperform their SigLIP counterparts and other open-weight baselines, with the B/16 variant achieving 78.2% ImageNet-1k zero-shot accuracy versus SigLIP’s 76.2%, and the So400m/14 variant reaching 84.1% at 384px resolution. The gains extend to localization tasks—referring expression comprehension improves from 67.33% to 86.04% on RefCOCO val for the L/16 256px model—and dense prediction tasks, where frozen SigLIP 2 representations yield 77.1% mIoU on PASCAL segmentation versus SigLIP’s 72.0%, establishing that a unified recipe combining decoder-based losses and self-supervised objectives produces vision encoders that excel across semantic understanding, localization, and dense feature extraction simultaneously, while a NaFlex variant further preserves native aspect ratios and supports multiple resolutions from a single checkpoint.

2. Context and Motivation

The Core Gap: A Unified Recipe That Doesn't Sacrifice Core Capabilities

The fundamental problem this paper addresses is that the field has produced a fragmented landscape of improvements to CLIP-style contrastive vision-language training, but no single open-weight model incorporates the full breadth of these advances. Each technique has been validated independently—often in isolation and sometimes on different base models, datasets, and benchmarks—but practitioners who want a production-ready vision encoder must choose which improvements to adopt and which capabilities to prioritize, because no prior release demonstrates that the techniques can coexist without mutual interference.

This matters because vision-language encoders serve as the foundational visual backbone for an increasingly diverse set of downstream applications. A model that excels at zero-shot classification (e.g., for content moderation) may produce subpar features for dense prediction tasks like segmentation (used in autonomous driving or medical imaging). A model optimized for English retrieval may fail catastrophically when deployed in a multilingual context (e.g., global search engines, educational tools in non-English-speaking regions). A model trained only with contrastive losses may struggle with localization tasks like referring expression comprehension (critical for visual question answering and embodied AI). The central tension is that improvements to one capability often come at the expense of others, and prior open releases force users to accept these tradeoffs rather than resolve them.

SigLIP 2's core contribution is demonstrating that these tensions are not fundamental—they can be resolved through careful recipe design. The paper's framing is explicitly about providing "the full breadth of latest improvements into a single model" that improves on core capabilities while also closing gaps in localization, dense features, and multilingual understanding.

Why the Problem Is Important: The Expanding Role of Vision Encoders

The practical significance of this gap has grown enormously because vision-language encoders are no longer just classifiers or retrieval engines. They are the visual perception component in Vision-Language Models (VLMs)—a use case the paper dedicates substantial evaluation to (Section 3.2). In this paradigm, a pretrained vision encoder (like SigLIP 2) feeds visual tokens into a Large Language Model (Gemma 2, in the paper's case), enabling the LLM to reason about images. The quality of the vision encoder directly determines the VLM's ability to answer questions, read text in images, count objects, ground referring expressions, and understand spatial relationships.

This expanding role creates new requirements that were never part of the original CLIP design brief:

• Dense feature quality: VLMs that need to segment objects or estimate depth cannot work with pooled image-level representations. They require per-patch features that carry spatial and semantic information—exactly the capability that standard contrastive training neglects because the contrastive loss operates on pooled representations (Section 2.3 explains that the sigmoid loss pools representations via the MAP head before computing alignment).
• Localization and grounding: Referring expression comprehension ("the red cup to the left of the laptop") requires the vision encoder to produce representations that support spatial reasoning. Vanilla CLIP-style models, trained only on image-level alignment, struggle here because they have no training signal that differentiates between spatial positions within an image.
• Multilingual understanding: As VLMs are deployed globally, the vision encoder must align visual concepts with text in many languages simultaneously. Training separate models per language is computationally prohibitive; a single multilingual vision encoder that works across languages is essential.
• Resolution flexibility: Different tasks need different resolutions. OCR requires fine-grained high-resolution processing, while natural image classification works well at lower resolutions. A model that requires separate checkpoints for each resolution forces painful engineering tradeoffs in deployment pipelines.

The paper's positioning, then, is that the field needs general-purpose vision backbones that perform well across all of these axes—not specialized models that excel at one and fail at another. SigLIP 2 is presented as a practical step toward that goal.

Prior Approaches and Their Limitations

The paper's related work section (Section 4) maps out the landscape of improvements to contrastive pretraining, each of which addresses a slice of the problem but leaves others unaddressed:

1. Re-captioning for Better Text Quality (Not Included in Prior Unified Releases)

Several works—LaCLIP, VeCLIP, and TIPS—showed that using VLMs to re-write or augment the alt-text captions used in training improves data quality and downstream performance. The intuition is straightforward: web-scraped alt-text is noisy, often ungrammatical, and sometimes completely unrelated to the image content. Better captions produce a cleaner training signal. However, re-captioning alone doesn't address dense feature quality or localization—it only improves the global image-text alignment signal. And importantly, no major open-weight model release (OpenCLIP, MetaCLIP, DFN, SigLIP v1) incorporated re-captioning as a standard component of their recipe.

2. Self-Supervised Auxiliary Losses (Validated in Isolation)

The combination of contrastive losses with self-supervised objectives—specifically local-to-global self-distillation (DINO-style consistency matching) and masked prediction (feature reconstruction at masked patch locations)—was shown by SILC and TIPS to dramatically improve dense prediction performance and even boost zero-shot classification and retrieval. The mechanism: contrastive losses push the model to produce semantically aligned patch features, while masked prediction forces the model to produce spatially informative patch features. Together, they produce representations that are both semantically meaningful (at the pooled level) and spatially structured (at the patch level).

However, SILC and TIPS were research papers, not model releases. They demonstrated the technique's effectiveness but didn't produce models that practitioners could download and use. Other open releases (OpenCLIP, MetaCLIP, EVA-CLIP, SigLIP v1) did not include these losses, meaning that users who wanted improved dense features had no open-weight option. The paper explicitly positions SigLIP 2 as filling this gap: providing models where these losses are integrated from the start.

3. Decoder-Based Pretraining for Localization (Separate from Contrastive Training)

Attaching a lightweight transformer decoder to the vision encoder and training it on captioning and referring expression tasks was explored by BLIP-2, CoCa, and LocCa. The decoder forces the vision encoder to produce patch-level representations that retain sufficient information for the decoder to generate words or bounding box coordinates—information that a purely contrastive loss (which only cares about the final pooled representation) might discard. This substantially improves localization capabilities (referring expressions, open-vocabulary detection) and OCR.

But prior work treated decoder-based pretraining largely as either:

• A standalone representation learning method (image captioners trained only with captioning loss, not contrastive)
• An additional component in a multi-stage pipeline where the contrastive loss and decoder loss are trained separately

No prior open-weight model integrated decoder-based pretraining into the standard contrastive training recipe. LocCa, for example, demonstrated strong localization results but used only decoder-based losses (no contrastive sigmoid loss), and it was trained on English-only data. SigLIP 2 incorporates the LocCa-style decoder losses alongside the sigmoid loss, making it the first open release to do so.

4. Data Filtering and Curation (The Dominant Focus of Prior Releases)

The most common improvement in prior open-weight releases has been data quality. MetaCLIP improved data filtering using metadata-based curation. DFN trained a filtering network on high-quality datasets and used it to select training examples. Both achieved strong classification and retrieval results by focusing on cleaner data. However, these approaches are purely data-centric—they don't modify the training loss or architecture, and consequently they don't address gaps in dense features or localization. DFN's strong ImageNet results (76.2% for B/16, matching SigLIP v1) come with an important caveat the paper explicitly flags: DFN's filtering network is fine-tuned on ImageNet, COCO, and Flickr—the exact benchmarks it's evaluated on—making it difficult to separate genuine representation quality from benchmark-specific optimization.

5. Multilingual Training (Sacrificing English Performance)

The original SigLIP paper included an mSigLIP variant trained on multilingual data. However, as the paper notes in Figure 2, mSigLIP achieves strong multilingual retrieval but "performs substantially worse than SigLIP and SigLIP 2 on English-focused benchmarks." This creates a frustrating tradeoff: you can have a model that works well in English, or a model that works adequately across many languages, but not both. SigLIP 2's stated goal is to nearly match mSigLIP's multilingual performance while exceeding SigLIP's English performance—eliminating the tradeoff.

6. Resolution Flexibility (Dominated by Distorted Resize Approaches)

The standard approach in CLIP-style training is to resize all images to a fixed square resolution, distorting aspect ratios. This is simple but problematic: a document image squeezed from 4:3 to 1:1 loses structural information critical for OCR. NaViT introduced native aspect ratio processing by packing variable-sized patches, and FlexiViT enabled a single ViT checkpoint to support multiple patch sizes/sequence lengths. But no prior release combined these ideas into a vision-language encoder—they were validated on image-only tasks (ImageNet classification for FlexiViT, image classification and detection for NaViT), not on the full suite of vision-language benchmarks.

How SigLIP 2 Positions Itself

The paper positions SigLIP 2 as a compositional contribution: the individual techniques are not novel (the paper is explicit that these are "prior, independently developed techniques"), but the act of combining them into a coherent, well-engineered recipe that produces complementary rather than conflicting improvements is genuinely new. The key intellectual move is recognizing that these techniques address different shortcomings of CLIP-style training:

• Contrastive loss (SigLIP): Produces strong global semantic alignment for classification and retrieval.
• Decoder-based losses (LocCa): Forces the vision encoder to retain spatial and textual information that the contrastive loss would discard, improving localization and OCR.
• Self-supervised losses (SILC/TIPS): Improves per-patch feature quality for dense prediction tasks, which the pooled contrastive loss doesn't directly optimize.
• Multilingual data mixture: Extends language coverage without the typical degradation in English performance.
• Data debiasing: Addresses fairness and representation bias, which is orthogonal to raw accuracy and requires explicit intervention.
• Active data curation (ACID): Compensates for the reduced training signal in small models by selectively training on the most informative examples.

The staged training approach (Section 2) is the engineering insight that makes this composition work: training with decoder losses from the start, adding self-supervised losses only at 80% of training completion (to avoid data augmentation interfering with image-text alignment), and then adapting resolutions in a final stage. This staging is not obvious—adding all losses simultaneously from the start could cause interference, as the self-supervised augmentations (strong cropping, color jittering) can degrade the image-text alignment signal.

The paper also explicitly addresses backward compatibility with SigLIP v1: "SigLIP 2 is designed to be backward compatible with SigLIP by relying on the same architecture. This allows existing users to simply swap out the model weights and tokenizer." This is a practical but important design choice—it means SigLIP 2 is a drop-in replacement, not a new architecture that requires pipeline changes (though users must switch to the multilingual Gemma tokenizer with 256k vocabulary).

Finally, the paper's release of four model sizes (ViT-B at 86M, L at 303M, So400m at 400M, and g at 1B parameters) reflects an understanding that different deployment scenarios have different compute budgets. The B/16 and B/32 models receive special treatment (distillation via active data curation in Section 2.5) because small models benefit disproportionately from training signal optimization—a finding consistent with the broader literature on model scaling but not previously applied to vision-language encoders in this way.

The Unresolved Tensions from Prior Work

A subtle but important motivation running through the paper is the conflict between data augmentation and vision-language alignment. Self-supervised methods like DINO derive much of their benefit from aggressive data augmentation (multi-crop, color jittering, Gaussian blur), but these same augmentations can destroy the semantic content that vision-language alignment depends on—strong color jittering can turn a "red car" image into something the text "red car" no longer describes accurately. The paper resolves this tension with a practical decision: "We use the original image for computing the SigLIP and LocCa losses... and apply the additional [self-supervised] losses on additional augmented views" (Section 2.3). This means the same training step processes both the unaugmented image (for text alignment) and augmented views (for self-supervision), allowing the model to learn from both signals simultaneously without interference. This solution is simple in retrospect but not obvious in advance, and it represents one of the key engineering insights that makes the unified recipe work.

3. Technical Approach

3.1 Reader Orientation

SigLIP 2 is a family of multilingual vision-language encoder models—specifically, Vision Transformer (ViT) image encoders paired with transformer text encoders—that take an image and a text string as input and produce dense vector representations (embeddings) capturing their semantic alignment. The system solves the problem of fragmented improvements to CLIP-style training by combining multiple independently-developed techniques into a single, coherent training recipe, producing models that simultaneously excel at zero-shot classification, image-text retrieval, dense prediction, localization, and multilingual understanding without requiring separate specialized checkpoints for each capability.

3.2 Big-Picture Architecture (Diagram in Words)

The SigLIP 2 system has five major components, each contributing a different training signal to the shared vision encoder:

1. Image Encoder (ViT) — the core component being trained, shared across all losses. Processes an input image into a sequence of patch-level feature vectors (un-pooled representation) plus a pooled global representation via a MAP (attention pooling) head. Available in four sizes: ViT-B (86M parameters), ViT-L (303M), So400m (400M), and ViT-g (1B).

2. Text Encoder (Transformer) — a matching transformer that processes text tokens (up to length 64, from the multilingual Gemma tokenizer with 256k vocabulary) into a pooled text embedding via its own MAP head. Trained jointly with the vision encoder through the sigmoid loss.

3. Sigmoid Loss (SigLIP v1) — the core image-text alignment objective. Treats every image-text pair in a batch as a binary classification problem: for each of the $B^2$ possible (image, text) pairs in a batch of size $B$, the model predicts whether the pair matches (positive) or not (negative). This produces the strong global semantic alignment needed for zero-shot classification and retrieval.

4. AR Decoder (LocCa) — a lightweight autoregressive transformer decoder with cross-attention that reads from the un-pooled vision encoder representations (before the MAP head). Trained on three auxiliary tasks simultaneously: standard image captioning, grounded captioning (predicting region-specific captions given bounding boxes), and referring expression prediction (predicting bounding box coordinates given region descriptions). This forces the vision encoder to retain spatial and textual information at the patch level.

5. Self-Distillation and Masked Prediction (SILC/TIPS) — two self-supervised losses added at 80% of training completion. The first enforces consistency between a student network (processing partial/local views of the image or masked versions) and a teacher network (processing the full image), using an EMA-updated teacher. The second specifically operates on masked patch positions, training the student to reconstruct the teacher's features at masked locations. These losses improve per-patch feature quality for dense prediction tasks.

Information flows as follows: An image enters the system and optionally receives augmentations (different views for different loss components). The base image (unaugmented) feeds into the vision encoder to produce features for the sigmoid loss (text alignment) and the LocCa decoder (captioning/localization). Augmented views (multi-crop, masking) feed through the same vision encoder (student) and are scored against the EMA teacher's representations for the self-supervised losses. The text encoder processes the caption to produce a pooled text embedding for the sigmoid loss and serves as the architecture template for the LocCa decoder. All gradients flow back through the shared vision encoder, which must learn to satisfy all objectives simultaneously.

3.3 Roadmap for the Deep Dive

• First, the training data, architecture details, and optimization setup (Section 2.1), because the data mixture and architecture choices—particularly the multilingual WebLI data with 109 languages, the 90/10 English/non-English split, the MAP head pooling, and the Gemma tokenizer—fundamentally shape what the model can learn and explain the multilingual and backward-compatibility properties.
• Second, the core sigmoid loss from SigLIP v1 (Section 2.2), since it is the primary image-text alignment mechanism that all other losses are built around, and understanding its binary classification formulation is essential for appreciating why dense features and localization are weak in vanilla SigLIP.
• Third, the decoder-based LocCa losses (Section 2.2), because they are added simultaneously with the sigmoid loss from the start of training and address the specific localization and OCR gaps that contrastive-only training creates.
• Fourth, the self-distillation and masked prediction losses from SILC/TIPS (Section 2.3), including why they are added only at 80% training completion, how the teacher EMA works, and the critical design decision to avoid augmenting the image used for the sigmoid and LocCa losses.
• Fifth, the resolution adaptation strategies (Section 2.4), covering both the fixed-resolution fine-tuning approach and the NaFlex variant that supports native aspect ratios and multiple resolutions from a single checkpoint.
• Sixth, the active data curation distillation for small models (Section 2.5), explaining the ACID method and the novel single-teacher variant that avoids explicit softmax distillation.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a systems and engineering paper whose core idea is that diverse, independently-developed improvements to CLIP-style training can be composed into a single recipe if the training is staged correctly and augmentation interference is handled explicitly. The result is not a single new technique but a demonstration that the techniques are complementary rather than conflicting.

---

Training Data, Architecture, and Optimization Setup

The foundation of the entire SigLIP 2 training recipe is the choice of data, model architecture, and optimizer—decisions that constrain and enable everything that follows. The paper makes specific, carefully-justified choices at each level.

Training data: WebLI with multilingual mixing. All models are trained on the WebLI dataset, which contains 10 billion images and 12 billion alt-text captions spanning 109 languages. The key design decision is the language mixture ratio: 90% of training image-text pairs are sourced from English web pages, and 10% from non-English web pages. This is not an arbitrary split—the paper explicitly cites prior work (Pouget et al., 2024) as recommending this ratio, and the motivation is to "strike a good balance between quality on English and multilingual vision-language benchmarks." A model trained on 100% English data would perform poorly on non-English retrieval; a model trained on a uniform language distribution would suffer degradation on English benchmarks (as mSigLIP did). The 90/10 split is the empirical compromise that nearly matches mSigLIP's multilingual performance (Figure 2) while exceeding SigLIP v1's English performance (Table 1).

The data is further processed with filtering techniques from Alabdulmohsin et al. (2024) to "mitigate data biases in representation and association with respect to sensitive attributes." These are the same debiasing filters that the fairness evaluation in Section 3.5 measures—they address both first-order biases (e.g., disparities in how often different genders appear) and second-order biases (e.g., biased associations between gender and occupation). The paper doesn't detail the filtering mechanism here (it references the prior work), but the effect is clear from Figure 6: SigLIP 2 L/16 at 256px has a representation bias of only 7.3%, compared to 35.5% for SigLIP v1 at the same size.

Architecture: Backward-compatible ViT with MAP head. The vision encoder follows the standard ViT architecture with learned positional embeddings, using the same design as SigLIP v1. This is explicitly justified for backward compatibility: "existing users can simply swap out the encoder weights." The text encoder uses the same transformer architecture as the vision encoder, except for the ViT-g model, which pairs a g-sized vision encoder with an So400m-sized text encoder (a compute-saving asymmetry—the text encoder doesn't need to scale as aggressively as the vision encoder for vision-language tasks).

Both vision and text representations are pooled using a MAP head (multi-head attention pooling), which is distinct from the standard CLS token approach used in the original ViT and CLIP. In MAP pooling, a set of learnable query vectors attend over the sequence of patch/text token features, producing a fixed-size pooled representation regardless of sequence length. The paper references the original MAP head work (Zhai et al., 2022) for details but doesn't re-explain the mechanism. The important implication: when the paper later discusses "un-pooled" versus "pooled" representations, the un-pooled representation is the full sequence of patch features before the MAP head aggregates them, and the pooled representation is the MAP head output. The self-supervised losses operate on the un-pooled features; the sigmoid loss operates on the pooled features.

Text tokenization: Multilingual Gemma tokenizer with 256k vocabulary. SigLIP 2 switches from the original SigLIP tokenizer to the Gemma tokenizer with a vocabulary size of 256,000 tokens. This is a substantial vocabulary size (typical English-only tokenizers are 30-50k), necessitated by the need to represent text in 109 languages efficiently. The text is lowercased before tokenization ("transforming the text to lower case before tokenization"), and the maximum text sequence length is set to 64 tokens. The large vocabulary has a computational cost—the paper mentions implementing a "chunked version of the decoder loss" specifically to reduce memory consumption from the large vocabulary when computing the LocCa decoder losses—but this is handled as an implementation detail rather than a fundamental design change.

Optimizer and training schedule. The paper uses Adam with a learning rate of $10^{-3}$, decoupled weight decay of $10^{-4}$, and gradient clipping to norm 1. The batch size is 32,000 (32k image-text pairs per training step). Training uses a cosine learning rate schedule with 20,000 warmup steps and continues for a total of 40 billion examples seen (40B). The hardware is up to 2048 TPUv5e chips using fully-sharded data-parallel training (FSDP).

These hyperparameters represent a specific engineering choice: the batch size of 32k is very large (typical image classification uses 256-4096), which is standard for contrastive training because the sigmoid loss benefits from many negative examples in each batch (every image-text pair in the batch that doesn't match is a negative). The learning rate of $10^{-3}$ with cosine decay and 20k warmup steps is a fairly standard recipe for large-scale transformer training, not specific to SigLIP.

Vision encoder configuration. Unless otherwise noted, all models use patch size 16 and image resolution 256×256 pixels, resulting in an image representation sequence length of 256 tokens (16×16 patches, each 16×16 pixels, covering a 256×256 image). The resolution and sequence length are varied in later adaptation stages (Section 2.4), but this is the base configuration from which all variants start.

---

The Sigmoid Loss: SigLIP's Core Contrastive Mechanism

The sigmoid loss from SigLIP v1 is the primary image-text alignment objective and remains the dominant training signal throughout the SigLIP 2 recipe. Understanding it is essential because its properties—specifically, that it operates on pooled representations—explain why the additional decoder and self-supervised losses are necessary.

How the sigmoid loss works. In each training step, a batch contains $B$ image-text pairs. The vision encoder produces $B$ image embeddings (pooled via the MAP head), and the text encoder produces $B$ text embeddings (also pooled via the MAP head). For every possible pair $(i, j)$ in the batch, the model computes the dot product similarity between the $i$-th image embedding and the $j$-th text embedding, applies a learnable temperature parameter $\tau$ and a learnable bias $b$, passes the result through the sigmoid function $\sigma(x) = 1/(1 + e^{-x})$, and is trained to output 1 for matching pairs (where $i = j$, i.e., the image and text come from the same training example) and 0 for non-matching pairs ($i \neq j$).

Formally, for a batch of $B$ pairs, the loss is:

$\mathcal{L}_{\text{sig}} = -\frac{1}{B^2} \sum_{i=1}^{B} \sum_{j=1}^{B} \left[ \mathbb{1}_{[i=j]} \log \sigma(s_{ij}) + (1 - \mathbb{1}_{[i=j]}) \log(1 - \sigma(s_{ij})) \right]$

where $\sigma(s_{ij})$ is the sigmoid of the scaled similarity for the $(i, j)$ pair, $\mathbb{1}_{[i=j]}$ is 1 when $i = j$ (positive pair) and 0 otherwise (negative pair), and the sum iterates over all $B^2$ possible pairwise combinations.

What it computes: For each of the $B^2$ image-text pairs in the batch, the model computes a scalar similarity score, maps it to a probability via the sigmoid function, and computes the binary cross-entropy against the ground-truth label (1 for matching, 0 for non-matching). The loss averages over all $B^2$ pairs. The learnable temperature $\tau$ controls the sharpness of the probability distribution—higher temperature produces softer probabilities, lower temperature produces more peaked (confident) predictions—and the learnable bias $b$ shifts the decision boundary.

Why this form over the standard CLIP contrastive loss: The standard CLIP loss uses a softmax over the batch dimension, which normalizes scores so they form a probability distribution across candidates. The sigmoid loss avoids this normalization, treating each pair independently as a binary classification problem. The key advantage is computational: the softmax normalization in CLIP requires all pairs to be compared to compute the denominator, which creates a coupling between batch size and loss behavior that complicates scaling. The sigmoid loss has no such coupling because each pair is independent—the loss for the $(i, j)$ pair does not depend on the similarity scores for $(i, k)$ pairs with $k \neq j$. Additionally, the sigmoid loss provides a per-pair confidence score that can be interpreted directly as a matching probability, whereas the softmax in CLIP produces relative scores that sum to 1 across candidates for each query.

The critical limitation that motivates SigLIP 2's additional losses: The sigmoid loss operates exclusively on pooled representations—the output of the MAP head, which aggregates information from all 256 patch tokens into a single global vector representing the entire image. This means the training signal provides no direct incentive for the model to produce high-quality per-patch features. The model can learn to route all semantically-relevant information through the pooling operation and treat the individual patch representations as disposable intermediates. This is why standard SigLIP models struggle with localization (which requires spatial information about where things are in the image) and dense prediction (which requires per-patch features that can be used for segmentation or depth estimation). The LocCa decoder and SILC/TIPS losses are both designed to address exactly this gap by providing training signals that operate on the un-pooled, per-patch features.

---

The LocCa Decoder: Adding Captioning and Localization Losses

The LocCa decoder addresses the localization gap in SigLIP by providing a training signal that requires the vision encoder to produce patch-level representations containing spatial and textual information. Unlike the sigmoid loss, which only needs the final pooled representation, the LocCa decoder cross-attends to the full sequence of un-pooled vision features and must extract specific information from specific spatial locations.

Decoder architecture. The LocCa decoder is a standard transformer decoder with cross-attention layers that attend to the vision encoder's output. It uses the same architecture as the text encoder, except: (1) cross-attention layers are inserted between the self-attention and feed-forward layers in each decoder block, and (2) the number of layers is reduced by a factor of two compared to the text encoder. This reduction is an engineering choice to manage memory and computation—the decoder runs three forward passes per training example (one for each task), and a full-depth decoder would make training prohibitively expensive. The decoder is trained from scratch alongside the vision encoder during the initial training phase (it is not pre-trained or frozen).

Three auxiliary tasks trained simultaneously. For each training image, the LocCa decoder produces predictions for three distinct tasks, each requiring a separate forward pass through the decoder (while the vision encoder forward pass is shared):

1. Image captioning: The decoder generates a textual description of the full image. This is trained with 50% probability using parallel prediction—all caption tokens are predicted simultaneously from mask tokens without a causal attention mask (each token attends to all other tokens). For the other 50% of examples, standard autoregressive prediction is used (each token is predicted given all previous tokens). Parallel prediction, introduced by the CapPa paper (Tschannen et al., 2023), is computationally efficient because all tokens can be processed in a single decoder forward pass rather than sequentially, and it provides a different learning signal—the model must predict each token without access to the tokens that would normally follow it in the sequence.

2. Automatic referring expression prediction: The decoder is given a caption describing a specific region of the image (e.g., "the red cup on the left") and must predict the corresponding bounding box coordinates. This forces the vision encoder to produce features that allow the decoder to spatially localize described objects, connecting the semantic content of the caption to specific positions in the image. The bounding boxes are represented as coordinate tokens in the decoder's output.

3. Grounded captioning: The complimentary task—the decoder is given bounding box coordinates and must generate a caption describing the corresponding image region. This forces the vision encoder to represent region-specific content in a way that the decoder can access and translate into language.

How region-caption pairs are generated. The training data for tasks 2 and 3 is created automatically from the WebLI alt-texts using the recipe from the OWL-ViT v2 paper (Minderer et al., 2023). The process first extracts n-grams from the alt-texts, then applies an open-vocabulary object detector to locate those n-grams in the image, producing region-caption pairs. Additionally, a fixed set of object categories from the WebLI paper (Chen et al., 2022) is used instead of n-grams for some examples, providing coverage for common objects that might not appear explicitly in alt-text n-grams. This automatic annotation means no human-annotated bounding boxes are needed, making the approach scalable to the billion-scale WebLI dataset.

Why the LocCa losses improve localization. The mechanism is straightforward but important: the sigmoid loss trains the vision encoder to produce a single pooled vector that captures "what is in the image" at a global level. The referring expression prediction task trains it to produce patch-level features that capture where specific things are. The grounded captioning task trains it to produce region-specific semantic representations. And the captioning task provides a bridge between the global and local representations by training the vision encoder to produce features that can be decoded into natural language. Together, these tasks create a gradient signal that propagates back through the vision encoder and shapes the patch-level features to be more spatially informative—the exact property that standard SigLIP lacks.

Chunked decoder loss for memory efficiency. Because the Gemma tokenizer has a vocabulary of 256,000 tokens, computing the full vocabulary softmax during decoder training would be memory-intensive (the output projection layer would have shape [hidden_dim, 256k]). The paper implements a "chunked version of the decoder loss" to reduce memory consumption—likely meaning the vocabulary is split into chunks and the loss is computed in segments, though the exact implementation details are not specified. This is presented as a necessary engineering optimization rather than a methodologically interesting choice.

The decoder is not part of the release. A critical point for understanding the model's use: "the decoder only serves for representation learning here and is not part of the model release." After training, the decoder is discarded. Only the vision encoder and text encoder weights are released. This means the LocCa training benefits are baked into the vision encoder's representations—downstream tasks don't need the decoder to benefit from improved localization, because the vision encoder has been trained to produce spatially-informative features. During evaluation on referring expression comprehension (Section 3.4.1), a new decoder is trained from scratch on the frozen vision encoder's features, demonstrating that the representations themselves have improved, not just the pretrained decoder.

---

Self-Distillation and Masked Prediction (SILC/TIPS)

The self-supervised losses address the dense feature quality gap—the fact that standard SigLIP produces per-patch features that, while useful for global pooling, lack the spatial structure and semantic richness needed for dense prediction tasks like segmentation and depth estimation. The paper adopts two complementary losses from the SILC and TIPS papers, which themselves build on the self-supervised learning line of work (DINO, DINOv2, iBOT).

Staged addition at 80% of training. The self-supervised losses are not present from the start of training. They are added at 80% of training completion, with the teacher initialized from the student parameters at that point and the additional parameters (MLP heads, mask token, corresponding optimizer state) initialized randomly. This staging is not arbitrary—it reflects a critical design insight about interference between data augmentation and vision-language alignment:

Self-supervised methods like DINO derive substantial benefit from strong data augmentations: random resized crops (which can crop out large portions of the image), color jittering (which shifts hue, saturation, and brightness), Gaussian blur, and solarization. These augmentations teach the model invariance to transformations that don't change semantic content. However, these same augmentations can degrade the image-text alignment signal: heavy color jittering might make a "red car" no longer look red; aggressive cropping might remove the object mentioned in the caption entirely. Adding these augmentations from the start of training would therefore create a conflict: the sigmoid loss would be trying to align text with images whose semantic content has been altered by augmentation.

The paper resolves this tension with a simple but effective design: the sigmoid and LocCa losses are always computed on the original, unaugmented image, while the self-supervised losses are computed on additional augmented views of the same image. This means in each training step, the vision encoder processes both the clean image (for text alignment) and multiple augmented versions (for self-supervision), learning from both signals simultaneously without interference. The authors explicitly cite SILC's recommendation: "We use the original image for computing the SigLIP and LocCa losses from the previous section and apply the additional losses on additional augmented views. This is done to ensure that data augmentation does not negatively impact the image-text alignment as recommended by [45]."

Why wait until 80%? The paper doesn't explicitly state the rationale, but the likely reason is practical: adding self-supervised losses from the start would mean the vision encoder is learning patch-level feature structure before it has converged to a reasonable global semantic representation. Starting at 80% allows the sigmoid loss to establish strong global alignment first, then the self-supervised losses refine the patch-level features without disrupting the already-learned global structure. The teacher initialization from the student at the 80% point ensures the teacher starts with strong representations rather than learning from scratch.

Loss 1: Local-to-global consistency (self-distillation). This loss, from SILC, trains the vision encoder (student) to match the representations of a teacher network across different views of the same image. Concretely:

• The teacher sees a single global view of the image (a crop covering a large portion of the image, with mild augmentation).
• The student sees 8 local views (smaller crops from random positions, with stronger augmentation).
• Both the student and teacher produce representations through a separate MLP projection head (not the MAP head used for the sigmoid loss).
• The loss encourages the student's representation of each local view to match the teacher's representation of the global view in the high-dimensional projection space.

The implementation uses one global teacher view and eight local student views, with augmentations and hyperparameters following the SILC paper exactly. The teacher's parameters are updated as an exponential moving average (EMA) of the student's parameters over previous iterations, a technique inherited from DINO and momentum contrastive learning that provides stable, slowly-evolving target representations.

The mechanism: By forcing the student to match the teacher's global representation from only a partial view, the model learns that local image patches contain information about the broader semantic context. A patch showing part of a dog's ear must carry information that it belongs to a "dog" context, not just a "furry texture" context. This encourages semantically meaningful patch-level features.

Loss 2: Masked prediction. This loss, from TIPS, specifically targets the quality of features at individual patch positions. The setup:

• Both student and teacher see the same global view of the image.
• In the student, 50% of the embedded image patches (50 of the 256 patch tokens at the base resolution) are replaced with a learnable mask token before being processed by the vision encoder.
• The loss is applied to features at the masked positions only, and matches the student's predicted features to the teacher's features at those same positions.
• The loss uses the same formulation as the local-to-global consistency loss (a feature matching objective in the projection space), but operates per-patch rather than on pooled representations.

The mechanism: The vision encoder must predict what the teacher would output at positions where the input is completely masked out. To do this successfully, it must use context from surrounding unmasked patches—the model learns spatial relationships and visual reasoning at the patch level. If patch 37 is masked but patches 21-36 and 38-52 show parts of a car wheel, the model must learn to reconstruct wheel-consistent features at position 37.

Loss weighting and per-model scaling. The two self-supervised loss terms are weighted differently: the first (local-to-global) has weight 1.0, and the second (masked prediction) has weight 0.25. These base weights are then further scaled by a model-size-dependent factor to "balance model quality on global/semantic and dense tasks": 0.25 for ViT-B, 0.5 for ViT-L, 1.0 for So400m, and 0.5 for ViT-g. This means the self-supervised losses have 4× less influence for the B model than for the So400m model. The paper doesn't explain the motivation for this scaling, but it likely reflects the observation that smaller models have less capacity to absorb additional training signals without degrading the primary sigmoid loss performance, while larger models have sufficient capacity to benefit from richer supervision. The So400m model, with its 1.0 factor, receives the strongest self-supervised signal relative to its primary loss.

---

Resolution Adaptation: Fixed-Resolution Fine-Tuning and NaFlex

Once the base model is trained at 256×256 resolution (sequence length 256, patch size 16), the paper applies two different strategies for supporting additional resolutions, depending on the use case.

Fixed-Resolution Adapted Variants

For users who need a specific resolution and can afford to deploy a separate checkpoint, the paper provides fixed-resolution variants at multiple sizes: 224×224, 256×256, 384×384, and 512×512 (with corresponding sequence lengths depending on patch size).

The adaptation procedure, distinct from standard fine-tuning. The paper uses an unusual approach: instead of fine-tuning the final checkpoint at a reduced learning rate (the "common strategy" they tried and found wanting), they resume training from the checkpoint at 95% of training completion (after the self-supervised losses have been applied for the final 15% of training, since they were added at 80%), resize the positional embeddings to the target sequence length, optionally resize the patch embedding (e.g., from patch size 16 to 14 using the pseudoinverse resize strategy from FlexiViT), and continue training at the target resolution with all losses active (sigmoid, LocCa, and self-supervised). The training runs for the remaining 5% of the original schedule at the new resolution.

The paper explicitly states that the standard fine-tuning approach—taking the final checkpoint, lowering the learning rate, and removing weight decay—"did not lead to good results across all sizes and resolutions." The alternative approach of continuing from 95% with all losses keeps the training dynamics consistent and ensures the model has time to adapt its positional embeddings and patch-level features to the new resolution while still receiving the full spectrum of training signals.

Pseudoinverse resize for patch embedding adaptation. When changing patch size (specifically, from patch size 16 to 14 for the L/14 and So400m/14 models), the patch embedding convolution kernel needs to be resized to process differently-sized patches. The pseudoinverse (PI) resize strategy from FlexiViT solves this by: (1) treating the original patch embedding as an interpolation function over the 16×16 patch, (2) evaluating this function at the positions corresponding to a 14×14 patch, and (3) solving a least-squares problem to find the 14×14 patch embedding that best approximates the original function's behavior on 14×14 patches. This produces better results than simpler interpolation methods (bilinear, bicubic) because it accounts for the spectral properties of the convolution operation.

NaFlex: Native Aspect Ratio and Flexible Resolution

NaFlex combines two ideas from prior work—FlexiViT's support for multiple sequence lengths with a single ViT, and NaViT's processing of images at their native aspect ratio—into a single variant that can handle images at arbitrary (well, discretely-predefined) resolutions and aspect ratios from one checkpoint.

Preprocessing for native aspect ratio. Given an input image with original dimensions $(H_{\text{orig}}, W_{\text{orig}})$, a target patch size $P$, and a maximum target sequence length $S_{\text{max}}$, NaFlex resizes the image such that:

1. The resized height $H_{\text{resized}}$ and width $W_{\text{resized}}$ are both multiples of the patch size $P$ (so the patch grid lands exactly on pixel boundaries).
2. The aspect ratio distortion is minimized (keeping $H_{\text{resized}} / W_{\text{resized}}$ as close as possible to $H_{\text{orig}} / W_{\text{orig}}$).
3. The resulting sequence length $N_{\text{patches}} = (H_{\text{resized}} / P) \times (W_{\text{resized}} / P)$ is at most $S_{\text{max}}$.

The paper notes that the distortion in width and height is at most $(P-1)/W_{\text{orig}}$ and $(P-1)/H_{\text{orig}}$ respectively, which is small for common resolutions. For example, with $P=16$ and a 1024-pixel-wide image, the maximum width distortion is $15/1024 \approx 1.5\%$. This is the same type of distortion incurred by NaViT's patch packing approach.

Handling variable sequence lengths in the ViT. Since images processed by NaFlex produce different numbers of patch tokens depending on their aspect ratio and resolution, the ViT must handle variable-length sequences. The paper's approach:

• A set of learned positional embeddings is defined with a fixed length of 256 (corresponding to a 16×16 patch grid). The positional embedding is interpreted as a continuous function over the 2D grid, and for any target grid shape $(h, w)$ where $h \times w \leq S_{\text{max}}$, the positional embedding is bilinearly resized (with anti-aliasing) to match the target grid. This means a 4:3 aspect ratio image producing a 12×16 grid would have positional embeddings sampled from the learned 16×16 grid at non-integer positions via interpolation.
• When the actual sequence length $N_{\text{patches}}$ is less than the target sequence length $S_{\text{max}}$ (which happens for images that don't use the full budget), padding tokens are added to reach $S_{\text{max}}$, and attention masks are applied in all transformer layers (including the MAP head) to prevent the model from attending to these padding tokens. This ensures the model's computation is unaffected by the presence of padded positions.
• Patch coordinates are added to the patch embeddings (as in NaViT) to provide explicit spatial information, and a mask indicating padding positions is included.

Training schedule for NaFlex. The NaFlex variant is initialized from the base SigLIP 2 checkpoint at 90% of training completion (not 95% like the fixed-resolution adapted variants), and specifically from a checkpoint trained with the setup from Section 2.2—meaning the self-supervised losses from Section 2.3 have not been applied yet. The paper then "switches to aspect-preserving resizing and uniformly sampling a sequence length from {128, 256, 576, 784, 1024} per mini-batch." The learning rate schedule for the final 10% of training is stretched by a factor of 3.75 to ensure each resolution receives sufficient training examples. Additionally, for the largest sequence length (1024), the batch size is halved and the number of training steps doubled to avoid out-of-memory errors.

Why NaFlex omits self-supervised losses. The paper explicitly states: "To keep implementation and computation complexity manageable, we do not apply self-distillation and masked prediction from Sec. 2.3." The reason is practical: the self-supervised losses require multiple augmented views per image (8 local student views plus a global teacher view for the consistency loss), and computing these for variable aspect ratios and resolutions at large batch sizes would significantly complicate the data pipeline and increase memory usage. The NaFlex variant therefore represents a tradeoff—it gains resolution flexibility at the cost of the dense feature improvements from SILC/TIPS. Figure 3 shows that for the So400m model, this tradeoff is essentially neutral (NaFlex matches standard SigLIP 2 on most benchmarks), but for the B/16 model, the standard variant generally outperforms NaFlex (likely because the B/16 benefits from the active data curation distillation, which NaFlex also omits).

---

Distillation via Active Data Curation for Small Models

The smallest models in the SigLIP 2 family—ViT-B/16 and ViT-B/32—receive an additional fine-tuning stage designed to maximize their performance given their limited capacity. The method, adapted from the ACID/ACED paper (Udandarao et al., 2024), uses a teacher model to guide data selection without explicit knowledge distillation.

The core problem with small models in contrastive training. Contrastive vision-language training at scale uses massive batches (32k) where each image sees thousands of negative text pairs. Small models with limited capacity may not benefit equally from all examples—some image-text pairs are trivially easy (the model already aligns them correctly), some are impossibly hard (the caption is noisy or the image content is too complex), and only a subset provide the right level of learning challenge. Randomly sampling from the full data distribution wastes training compute on examples the model has already mastered. Active data curation selects examples that are maximally informative for the current model state.

The ACID method (Active Data Curation via Implicit Distillation). The core mechanism operates at each training step:

1. A super-batch of size 64k (for filtering ratio 0.5) or 128k (for filtering ratio 0.75, used for B/32) is sampled from the training data.
2. Both the teacher model and the current learner model score each example in the super-batch by its "learnability"—a measure of how much the learner would benefit from training on that example, defined in the referenced prior work (Mindermann et al., 2022) as points that the model currently gets wrong but has the capacity to learn, balancing between "too easy" (already correct) and "too hard" (beyond the model's capacity).
3. The top-scoring examples are selected to form the actual training batch of size 32k.
4. Training proceeds on this curated batch using only the sigmoid image-text loss (the LocCa decoder and self-supervised losses are not used during this stage).

The filtering ratio determines the super-batch size relative to the training batch: a ratio of 0.5 means the super-batch is 64k (double the training batch of 32k), and 50% of examples are filtered out. A ratio of 0.75 means the super-batch is 128k (quadruple), and 75% are filtered. The B/32 model uses the higher ratio because its smaller capacity makes data quality more critical.

Fine-tuning configuration for this stage. The learning rate is reduced to $10^{-5}$ (100× lower than the main training), weight decay is removed entirely, and training continues for an additional 4 billion examples using only the sigmoid image-text loss on the curated batches. The reduced learning rate and removed weight decay are standard fine-tuning practices to avoid catastrophic forgetting of the representations learned during the main training phase.

The novel single-teacher variant: avoiding explicit distillation. The original ACED method (ACID + Explicit Distillation) uses two teacher models—one trained on diverse data (to provide broad concept coverage) and another trained on a curated high-quality dataset (to guide toward clean, well-aligned examples)—and applies explicit softmax distillation from teacher to student. The paper proposes a simplification: instead of two teachers with explicit distillation, they take a "single strong teacher trained on the diverse data (in this case, the SigLIP 2 So400m model) and fine-tune it for 1B examples on the high-quality curated dataset from [Evans et al., 2024]." This fine-tuned teacher is then used in the ACID method as described above.

The key insight is that the teacher, by being fine-tuned on high-quality curated data, develops a preference for examples that align with the curated data distribution. The ACID example selection, by picking examples the teacher considers "learnable," implicitly transfers this preference to the student—the student sees examples that the high-quality-aware teacher identifies as informative. This achieves the benefit of explicit distillation (guiding the student toward high-quality data) without the computational cost of running softmax distillation losses. The paper states: "Because this teacher blends diverse knowledge of concepts from pretraining, with knowledge of what is high-quality (from the curated dataset), the implicit distillation of ACID alone is sufficient to recover the benefits of ACED."

Why this matters for the B-sized models. The active data curation stage is only applied to B/16 and B/32—the smallest models. The L, So400m, and g models do not undergo this additional fine-tuning. The reason is that larger models have sufficient capacity to absorb information from randomly-sampled data without overfitting to noise; the benefit of curation diminishes as model capacity increases because large models can "average out" noisy examples across many parameters. Small models, with limited capacity, benefit disproportionately from seeing higher-quality, more informative examples. Table 1 shows the result: SigLIP 2 B/16 reaches 78.2% ImageNet-1k zero-shot compared to SigLIP's 76.2%, and SigLIP 2 B/32 reaches 74.0% versus OpenCLIP's 72.8%—gains that the paper attributes partly to this distillation stage.

---

Summary of Design Choices and Their Justifications

• 90/10 English/non-English data split (not 50/50, not 100/0): empirically balances English benchmark performance with multilingual retrieval quality, avoiding the performance degradation that mSigLIP suffered on English tasks.
• Original image for sigmoid/LocCa, augmented views for self-supervised losses: prevents augmentation from corrupting the image-text alignment signal while still allowing the self-supervised losses to benefit from strong augmentations that teach invariance.
• Self-supervised losses added at 80% of training: allows the global image-text alignment to converge before introducing patch-level objectives, preventing training instability from conflicting early-stage gradients.
• Continued training from 95% checkpoint for resolution adaptation rather than fine-tuning final checkpoint: standard fine-tuning (lower LR, no weight decay) "did not lead to good results across all sizes and resolutions"; continuing with all losses active maintains training dynamics consistency.
• MAP head pooling instead of CLS token: inherited from SigLIP v1 for backward compatibility; attention pooling can adaptively weight different spatial positions, potentially capturing more nuanced global representations than a single CLS token.
• Chunked decoder loss: necessary engineering optimization to handle the 256k vocabulary of the Gemma tokenizer within memory constraints, not methodologically motivated.
• ACID with single fine-tuned teacher instead of ACED with two teachers and explicit distillation: reduces computational cost while preserving the data curation benefit through implicit knowledge transfer via example selection.
• Model-size-dependent self-supervised loss weighting (0.25 for B, 0.5 for L, 1.0 for So400m, 0.5 for g): compensates for the fact that smaller models have less capacity to absorb additional training signals without degrading primary task performance; the So400m, with 1.0, receives the strongest self-supervised signal.
• NaFlex omits self-supervised losses and distillation: pragmatic tradeoff to keep implementation complexity manageable while still providing resolution flexibility; the fixed-resolution variants remain the higher-quality option when deployment constraints allow separate checkpoints.

4. Key Insights and Innovations

Innovation 1: A Unified Training Recipe That Demonstrates Losses Are Complementary, Not Conflicting

The paper's most significant intellectual move is not proposing any new loss function or architecture, but rather demonstrating—through careful engineering and staged training—that several independently-developed auxiliary objectives can coexist without the destructive interference that the field implicitly assumed would occur. Prior to SigLIP 2, the dominant pattern in open-weight vision-language model releases was conservatism: almost all major releases (OpenCLIP, MetaCLIP, EVA-CLIP, DFN, SigLIP v1) stuck close to the original CLIP contrastive formula, with improvements focused almost exclusively on data quality and filtering. Even work that proposed richer training signals—SILC's self-distillation, TIPS's spatial awareness losses, LocCa's decoder-based pretraining, CapPa's parallel prediction—validated these methods in research settings but did not produce integrated open-weight models that practitioners could use.

The unspoken assumption this conservatism reflected was that combining these techniques would cause interference: self-supervised augmentations might corrupt image-text alignment; decoder losses might compete with contrastive objectives for the vision encoder's representational capacity; adding too many objectives simultaneously might destabilize training. SigLIP 2's core contribution is proving these fears are manageable. The evidence is the consistent Pareto improvement across task families that were previously treated as involving unavoidable tradeoffs. Specifically:

• Localization versus global semantics (Table 5 vs. Table 1): SigLIP 2 L/16 256px improves RefCOCO val referring expression comprehension from SigLIP's 67.33% to 86.04%—a 18.7 percentage point gain—while simultaneously improving ImageNet-1k zero-shot classification from 80.5% to 82.5%. These capabilities were previously seen as competing; SigLIP 2 shows they can be jointly improved.
• Dense features versus image-text retrieval (Table 2 vs. Table 1): SigLIP 2 So400m/14 224px achieves 77.1% mIoU on PASCAL segmentation (up from SigLIP's 72.0%) while also reaching 83.2% ImageNet-1k (up from 82.2%). The self-supervised losses improve dense features without degrading global alignment.
• Multilingual coverage versus English performance (Figure 2 vs. Table 1): SigLIP 2 nearly matches mSigLIP's cross-lingual retrieval on Crossmodal-3600 while substantially exceeding SigLIP v1's English-focused performance—something mSigLIP failed to do. The 90/10 language mixture ratio resolves a tradeoff that the original multilingual SigLIP variant could not address.

What makes this intellectually distinctive is not any individual technique but the compositional claim: these techniques target different deficiencies in standard contrastive training, and the improvements are complementary rather than overlapping. The sigmoid loss provides global semantic alignment; the LocCa decoder forces retention of spatial information that contrastive training would otherwise discard; the self-supervised losses enforce patch-level feature quality for dense prediction tasks; and the multilingual data mixture extends language coverage through a separate axis entirely. The paper's staged training approach (decoder from the start, self-supervised at 80%, separate augmentation pipelines per loss type) is the engineering insight that makes composition work, but the intellectual contribution is the reframing of these techniques from "alternative training paradigms" to "orthogonal axes of improvement."

This is a fundamental shift in how the field should think about vision-language pretraining: it is not a choice between contrastive alignment, captioning, and self-supervision—it is an AND, and the practical recipe exists to realize the combination.

Innovation 2: Difficulty-Conditioned Loss Scheduling as a General Principle for Multi-Objective Pretraining

A more specific but equally important insight embedded in the training recipe is the idea that different objectives have different optimal "windows" during training, and that staging losses based on training progress resolves conflicts that would occur if all objectives were trained simultaneously from initialization. While the paper presents this as an engineering decision rather than a theoretical contribution, it represents a diagnosis that generalizes beyond SigLIP 2.

The key observation is that self-supervised objectives (local-to-global consistency, masked prediction) benefit from strong data augmentations—multi-crop, color jittering, blur—but these same augmentations degrade the image-text alignment signal that the sigmoid loss depends on. Prior work that naïvely combined contrastive and self-supervised losses (SLIP, for example) applied the same augmented images to both objectives, forcing a compromise. SigLIP 2's solution—using unaugmented images for the sigmoid and LocCa losses and separate augmented views for the self-supervised losses—is a practical fix that the paper explicitly attributes to SILC's recommendation (Section 2.3: "This is done to ensure that data augmentation does not negatively impact the image-text alignment as recommended by [45]").

But the deeper insight is the temporal staging: the self-supervised losses are added at 80% of training completion, not from the start. The paper does not ablate this decision or provide a theoretical justification, but the logic is clear from the architecture: the sigmoid loss establishes global semantic structure first, giving the vision encoder a coherent representational space, and then the self-supervised losses refine patch-level features within that established space. Adding self-supervision from the start would force the model to simultaneously learn what makes a good global representation (from the contrastive signal) and what makes a good local representation (from the self-distillation signal), with gradients from the two objectives potentially pulling in different directions before the representational space is stable.

This temporal conditioning—"train global objectives first, then introduce local refinement"—is an inference-time analog of curriculum learning but applied to multi-objective pretraining, and it likely explains why prior attempts at combining these losses produced suboptimal results. The field's default assumption has been that auxiliary losses should be present throughout training (as in SLIP, CoCa, and BLIP-2), but SigLIP 2's results suggest that for some loss combinations, late introduction is superior because it avoids early-stage gradient conflict while still allowing the auxiliary signal to shape the final representation.

The evidence for this staging's importance is indirect (the paper never ablates the 80% decision), but the broader pattern of improved performance across both global and local metrics—rather than a boost in one at the expense of the other—is consistent with the staging explanation. This insight is incremental in its immediate application but fundamental in its implications for multi-objective pretraining design: the optimal training recipe may involve not just which losses to include but when to introduce them.

Innovation 3: Implicit Distillation Via Active Data Curation as a Dropped-in Replacement for Explicit Knowledge Distillation

In Section 2.5, the paper makes a methodological contribution that is easy to overlook because it is presented as a practical optimization for small models, but which represents a genuinely novel finding: active data curation with a single fine-tuned teacher can match or exceed the performance of explicit knowledge distillation with multiple teachers, while being computationally cheaper.

The context: The ACED method (Udandarao et al., 2024) proposed combining active data curation (ACID) with explicit softmax distillation, using two teacher models—one trained on diverse data for broad concept coverage, and one trained on a high-quality curated dataset to guide the student toward clean examples. This two-teacher setup with explicit distillation achieved the best reported performance but at significant computational cost (the softmax distillation requires running both teacher and student forward passes on the full batch, and storing both models in memory).

SigLIP 2's simplification—taking a single So400m teacher that was first pretrained on diverse data and then fine-tuned on the high-quality curated dataset, then using it only for example selection (not explicit distillation)—achieves qualitatively similar benefits. The mechanism is clever: the fine-tuned teacher, having been exposed to both diverse and curated data, has developed a preference for examples that are both informative (from its diverse pretraining) and high-quality (from its curated fine-tuning). The ACID selection process, by picking examples the teacher considers "learnable," implicitly transfers this dual preference to the student—the student trains on examples that satisfy both criteria, without ever explicitly matching the teacher's output distribution.

The intellectual significance of this is twofold. First, it suggests that explicit knowledge distillation may be unnecessary when the data selection process is guided by a teacher that already encodes the desired model properties—the selection mechanism serves as an implicit form of distillation that transfers teacher knowledge through the choice of training examples rather than through output matching. Second, it demonstrates that the teacher does not need to be a separate model trained on different data; a single teacher with a multi-stage training history (diverse pretraining followed by curated fine-tuning) can serve both purposes simultaneously.

This is a fundamental insight about the relationship between data curation and distillation: they are not separate techniques but can be collapsed into a single mechanism when the teacher's data preferences are properly aligned with the desired student behavior. The computational savings are substantial—explicit softmax distillation over 4B examples at batch size 32k would require roughly double the forward passes of ACID alone, plus the memory overhead of maintaining teacher model weights alongside student weights. The paper's single-teacher ACID variant achieves the same effect with no additional forward passes beyond the curation step.

The evidence is in the B/16 and B/32 results in Table 1, where SigLIP 2 small models significantly outperform their SigLIP v1 counterparts and rival much larger baselines (the B/32 achieves 74.0% ImageNet-1k, surpassing the 72.8% of OpenCLIP's B/32). While the paper does not provide an ablation comparing ACID with and without the fine-tuned teacher, the explicit claim is that "the implicit distillation of ACID alone is sufficient to recover the benefits of ACED" when the teacher is properly prepared.

Innovation 4: Verifier-Style Diagnostic of Aspect Ratio Distortion Effects via NaFlex

The NaFlex variant (Section 2.4.2) is, at the surface level, an engineering contribution—a way to train a single model checkpoint that supports multiple resolutions and native aspect ratios. But the paper deploys NaFlex as a diagnostic tool that reveals something surprising about how aspect ratio distortion affects different task families, and this diagnostic finding is the actual intellectual contribution.

Figure 3 and Table 7 compare NaFlex against the standard fixed-resolution SigLIP 2 variants across a range of benchmarks spanning natural image retrieval (COCO, Flickr), text-heavy retrieval (TextCaps, HierText, SciCap, Screen2Words), and standard classification (ImageNet variants). The diagnostic finding is that aspect ratio distortion has heterogeneous effects across task types, and these effects are not captured by standard evaluation protocols that focus exclusively on natural images:

• On text-heavy and document-focused benchmarks (HierText, SciCap, Screen2Words), NaFlex consistently outperforms the standard square-input variant, particularly at small sequence lengths where distortion is most severe. At sequence length 256, NaFlex So400m achieves 29.8% T→I retrieval on SciCap versus 29.7% for the standard variant—but at sequence length 64, the gap widens dramatically (11.8% vs. 5.2% on SciCap T→I for B/16).
• On natural image benchmarks (COCO, Flickr, ImageNet), the standard variant often matches or exceeds NaFlex, especially for the B/16 model where the standard variant benefits from the active data curation stage that NaFlex omits.

This finding matters because the dominant paradigm in vision-language model evaluation—and by extension, in model development—has been to optimize for natural image benchmarks and treat aspect ratio handling as an implementation detail. The NaFlex results show that this approach systematically underestimates the cost of aspect ratio distortion on document understanding, OCR, and UI-focused tasks. A model that achieves 78.2% on ImageNet-1k (standard B/16 at 224px) might achieve only 5.5% T→I retrieval on HierText at the same resolution, whereas a native-aspect-ratio variant could achieve 8.7%—a 58% relative improvement on the text-heavy benchmark that is invisible to standard evaluation.

The broader implication is that the research community's focus on fixed-resolution, natural-image benchmarks has created a selection pressure that favors models which tolerate aspect ratio distortion well, while disadvantaging approaches that would perform better on the increasingly important class of document and screen-based tasks. This is not a criticism of prior work but an observation about the implicit priorities encoded in standard benchmarks: if your evaluation suite consists primarily of COCO, Flickr, and ImageNet, you will never discover that your square-input model is substantially suboptimal for document understanding, because you never measure it.

This diagnostic use of NaFlex is a genuinely novel contribution—it is not simply "we built a flexible model" but "we built a flexible model and used it to reveal that standard practices are more damaging to certain task families than previously recognized." The finding is incremental in its immediate practical impact (NaFlex is a variant, not the primary model family) but fundamental in its implications for evaluation design and model development priorities.

5. Experimental Analysis

Evaluation Methodology

• Datasets. The primary benchmarks span several task families. For zero-shot classification: ImageNet-1k (1,000 classes), ImageNet-v2 (matched frequency), ImageNet ReaL (relabeled), and ObjectNet (bias-controlled). For image-text retrieval: COCO (5,000 test images), Flickr30k (1,000 test images), and Crossmodal-3600 (XM3600, covering 36 languages). For dense prediction: PASCAL segmentation, ADE20k, NYUv2 depth, and NAVI normals. For localization: RefCOCO, RefCOCO+, and RefCOCOg variants. For cultural diversity: Dollar Street (21K images mapped to ImageNet classes), GeoDE (geographically diverse), and GLDv2 (landmark retrieval). The evaluation uses standard test splits for each benchmark as specified in the original dataset publications.

• Base model(s). SigLIP 2 models span four sizes: ViT-B (86M parameters), ViT-L (303M), So400m (400M), and ViT-g (1B). All use ViT architectures with MAP head pooling and patch size 16 unless otherwise noted (L/14 and So400m/14 variants use patch size 14 with pseudoinverse resize). The base training resolution is 256×256 (sequence length 256), with adapted variants at 224, 384, and 512 pixels. The text encoder matches the vision encoder architecture except for the ViT-g variant, which pairs a g-sized vision encoder with an So400m-sized text encoder. The choice to span four model sizes enables comparison of scaling trends across resolutions and model capacity.

• Metrics. Zero-shot classification uses top-1 accuracy (percentage of test images for which the highest-scoring class label is correct). Retrieval uses recall@1 (the fraction of queries for which the top-ranked result is correct), reported separately for text-to-image (T→I) and image-to-text (I→T). Semantic segmentation uses mean Intersection-over-Union (mIoU). Depth estimation uses Root Mean Square Error (RMSE, lower is better). Surface normal estimation uses angular RMSE (lower is better). Referring expression comprehension uses Acc@0.5 (the fraction of predictions where the predicted bounding box has IoU ≥ 0.5 with the ground truth). Open-vocabulary detection uses Average Precision (AP), with LVIS reporting AP for rare categories (APr) separately. Fairness uses representation bias (the tendency to associate random objects with a particular gender group, expressed as a percentage; lower is better) and performance disparity (maximum difference in accuracy across Dollar Street income levels).

• Baselines. The paper compares against an extensive set of prior open-weight vision-language models, all at comparable or larger model sizes and resolutions: CLIP (Radford et al., 2021, the original contrastive model), OpenCLIP (Ilharco et al., 2021, open-source reproduction at multiple scales), MetaCLIP (Xu et al., 2024, data curation via metadata filtering), EVA-CLIP (Sun et al., 2023, improved training techniques), SigLIP v1 (Zhai et al., 2023, the direct predecessor using sigmoid loss), DFN (Fang et al., 2024, data filtering network fine-tuned on evaluation benchmarks), CLIPA-v2 (Li et al., 2023, efficient CLIP training), and mSigLIP (SigLIP v1's multilingual variant). For the VLM transfer experiments, AIMv2 (Fini et al., 2024, a recent captioner-based vision encoder) is included. For referring expression comprehension, Cap and CapPa (Tschannen et al., 2023, pure image captioners) and LocCa (Wan et al., 2024, location-aware captioner) are included. Where baseline numbers are available, they are quoted directly from prior publications or reproduced by the authors.

• Generation budget / compute accounting. All models are evaluated in a zero-shot or frozen-encoder transfer setting—there is no test-time compute scaling budget to account for. The "compute budget" in this paper refers to training compute: all models are trained for 40 billion examples at batch size 32,000, with the NaFlex variant receiving an extended schedule (the final 10% stretched by 3.75×). The key fairness consideration in comparisons is model size and input resolution: Table 1 groups results by ViT variant and sequence length to enable matched comparisons. The FLOPs of different models at the same size and resolution are roughly comparable (since architecture is identical), making the primary variables training recipe and data rather than inference cost.

• Cross-validation / statistical protocol. The paper does not report confidence intervals, standard deviations, or cross-validation for the main benchmark results. For the compute-optimal search and revision experiments, there is no test-time strategy selection that would require cross-validation, since all models are evaluated in a standard zero-shot or frozen-encoder regime without per-example adaptation. The cultural diversity and fairness evaluations report results across disaggregated subgroups (income levels, geographic regions) without statistical testing for significance of differences. This absence of statistical rigor is a limitation—particularly for the fairness and cultural diversity results where subgroup differences could be noisy due to small per-group sample sizes—but is consistent with standard practice in the vision-language model literature where benchmark results are reported as point estimates on fixed test sets.

---

Main Quantitative Results

Zero-Shot Classification and English Retrieval

Headline results for B/16 at 224px (Table 1): SigLIP 2 achieves 78.2% ImageNet-1k zero-shot accuracy, compared to 76.2% for SigLIP v1, 76.2% for DFN, 74.7% for EVA-CLIP, 72.4% for MetaCLIP, 70.2% for OpenCLIP, and 68.3% for the original CLIP. This 2.0 percentage point improvement over SigLIP v1 may appear modest in absolute terms, but it comes despite the model being multilingual—SigLIP v1 was English-only—and represents the highest reported zero-shot accuracy for a B/16 model that does not use benchmark-specific data filtering (unlike DFN, which fine-tunes a filtering network on ImageNet, COCO, and Flickr).

Effect of resolution on classification (Table 1, B/16 column): Scaling resolution from 224px (sequence length 196) to 256px (sequence length 256), SigLIP 2 improves from 78.2% to 79.1% on ImageNet-1k, and further to 80.6% at 384px (576 tokens) and 81.2% at 512px (1024 tokens). SigLIP v1 follows a similar trajectory but consistently trails: 76.2% → 76.7% → 78.6% → 79.2%. By 512px, SigLIP 2's lead widens to 2.0 percentage points. The same pattern holds on ImageNet-v2 (74.5% vs. 72.9% at 512px), ImageNet ReaL (86.7% vs. 84.9%), and ObjectNet (77.8% vs. 74.8%).

Retrieval improvements (Table 1): On COCO text-to-image retrieval at 256px, SigLIP 2 B/16 achieves 53.2% recall@1 versus SigLIP v1's 47.4%. This 5.8 percentage point gap is substantially larger than the classification gains, suggesting the LocCa decoder losses (which train the vision encoder to produce spatially informative features) particularly benefit retrieval, which requires fine-grained matching between image regions and text queries. The image-to-text retrieval gap is similarly large: 69.7% vs. 65.1%. On Flickr30k, the gaps are comparable: 81.7% vs. 78.3% (I→T) and 94.4% vs. 91.1% (T→I).

Scaling with model size (Table 1, comparing B/16 → L/16 → So400m/14 → g/16 at similar resolutions): At 256px (or 224px for So400m/14), ImageNet-1k zero-shot accuracy scales as follows: SigLIP 2 B/16 = 79.1%, L/16 = 82.5%, So400m/14 = 83.2%, g/16 = 84.5%. The corresponding SigLIP v1 numbers are B/16 = 76.7%, L/16 = 80.5%, So400m/14 = 82.2%—the ViT-g was not available in SigLIP v1. The L/16 model shows the largest absolute improvement over SigLIP v1 at this resolution (2.0 percentage points), while the relative improvement is similar across scales (roughly 2-3%).

The B/32 model's surprising strength (Table 1): SigLIP 2 B/32 at 256px achieves 74.0% ImageNet-1k, substantially outperforming MetaCLIP B/32 at 224px (67.7%) and OpenCLIP B/32 at 256px (72.8%). This is notable because the B/32 model processes images at very low resolution (only 64 patches from a 256×256 image)—the active data curation distillation (Section 2.5) appears to be particularly effective at this scale, providing the model with a stronger training signal that compensates for the reduced visual information.

Multilingual Retrieval

Headline finding (Figure 2): SigLIP 2 achieves multilingual retrieval performance close to mSigLIP (the 100% multilingual-trained variant) while dramatically exceeding SigLIP v1 (English-only). As the paper states, "SigLIP 2 almost matches the performance of mSigLIP despite performing substantially better on English vision-language tasks (Table 1)."

Per-language breakdown (Figure 2, So400m/16 256px): The gap between SigLIP 2 and mSigLIP varies by language. For high-resource languages (French, German, Spanish, Italian), SigLIP 2 matches or exceeds mSigLIP—for example, on French text-to-image retrieval, all three models cluster around 0.48-0.50 recall@1. For lower-resource languages (Maori, Telugu, Cusco Quechua), SigLIP 2 trails mSigLIP more significantly—on Maori T→I, mSigLIP achieves approximately 0.15 while SigLIP 2 reaches approximately 0.10. Across all 36 languages in Crossmodal-3600, the average T→I recall@1 for SigLIP 2 is 46.5% (L/16 256px) and the average I→T is 56.5% (Table 1). The corresponding numbers for SigLIP v1 are 30.9% T→I and 40.1% I→T—a massive 15.6 and 16.4 percentage point improvement respectively. This demonstrates that the 90/10 English/non-English data mixture (Section 2.1) successfully captures multilingual capabilities without the English degradation that plagued mSigLIP.

Crossmodal-3600 in Table 1: The exact values for the So400m/16 256px variant are T→I 48.1% and I→T 57.5% for SigLIP 2, versus mSigLIP at 50.0% and 62.8%. SigLIP v1 scores 30.9% and 40.1%—the model essentially fails on non-English retrieval because it was never trained on non-English data. The interesting comparison is SigLIP 2 versus mSigLIP: mSigLIP holds a 1.9 percentage point advantage in T→I and a 5.3 point advantage in I→T, but achieves this at the cost of substantially worse English performance (mSigLIP So400m/16 256px: 80.8% ImageNet-1k vs. SigLIP 2's 83.4%). SigLIP 2 recovers most of mSigLIP's multilingual capability while adding 2.6 percentage points of ImageNet accuracy—a tradeoff that strongly favors SigLIP 2 for applications where English performance matters.

NaFlex vs. Fixed-Resolution Variants

The diagnostic value of resolution-by-resolution comparison (Figure 3, Table 7): The experiment compares a single NaFlex checkpoint (supporting native aspect ratio at multiple sequence lengths) against separate fixed-resolution SigLIP 2 checkpoints (each trained for a specific square resolution). The x-axis shows sequence length; for NaFlex, this is the training sequence length that was sampled during NaFlex training; the model is evaluated at multiple lengths.

On text-heavy benchmarks, NaFlex dominates at low resolutions (Figure 3, Table 7): At sequence length 64 (the smallest evaluated), NaFlex B/16 achieves 6.4% T→I and 11.0% I→T on Screen2Words, while the standard square-input B/16 is not evaluated at this resolution (the smallest standard variant is at 196 tokens for 224px). On HierText at sequence length 256, NaFlex B/16 reaches 7.4% T→I vs. the standard variant's 6.1%—a 21% relative improvement. On SciCap at sequence length 256, NaFlex B/16 achieves 19.7% T→I vs. standard's 17.1%. The pattern is consistent: for OCR, document, and screen understanding tasks, preserving the native aspect ratio improves retrieval quality, and the benefit is largest when the total resolution budget is small (because distortion is more severe when fewer pixels are available).

On natural image benchmarks, the standard variant often wins (Figure 3, Table 7): For COCO T→I at 256 tokens, standard B/16 achieves 53.2% vs. NaFlex B/16's 51.1%. For ImageNet-1k at 256 tokens, standard B/16 scores 79.1% vs. NaFlex's 78.5%. The paper attributes this to the standard variant benefiting from the active data curation stage (Section 2.5) and the self-supervised losses (Section 2.3), both of which NaFlex omits to "keep implementation and computation complexity manageable."

At higher resolutions, the gap narrows (Figure 3, Table 7): By sequence length 1024, the standard and NaFlex B/16 variants are nearly identical on natural image benchmarks: COCO T→I is 55.2% (standard) vs. 52.9% (NaFlex). On text-heavy benchmarks at high resolutions, NaFlex maintains or extends its lead: HierText T→I at 1024 tokens is 8.1% (standard) vs. 9.1% (NaFlex), and SciCap T→I is 25.2% (standard) vs. 24.8% (NaFlex)—the gap narrows on SciCap but NaFlex still leads. For the So400m model, the differences between NaFlex and standard are even smaller: at 1024 tokens, COCO T→I is 56.0% (standard) vs. 55.8% (NaFlex), and SciCap T→I is 35.9% (standard) vs. 32.6% (NaFlex)—here the standard variant actually leads on SciCap at high resolution, suggesting that for larger models, the benefit of native aspect ratio diminishes as resolution increases, since distortion becomes less severe in absolute terms.

The paper's own interpretation (Section 3.1.1): "The NaFlex variant outperforms the standard variant on the majority of these retrieval benchmarks, in particular for small sequence lengths (and hence resolutions) which tend to suffer more from aspect ratio distortion." This is a sober assessment: NaFlex is not uniformly better but is specifically valuable when operating at limited resolution budgets on text-heavy content—exactly the deployment scenario where aspect ratio distortion is most damaging.

Vision Encoder for VLMs (PaliGemma Transfer)

Experimental setup (Section 3.2, Figure 4, Table 6): A frozen SigLIP 2 vision encoder is paired with a Gemma 2 2B LLM and trained on 50 million examples from the PaliGemma stage 1 mixture (captioning, OCR, grounded captioning, visual question answering, detection, instance segmentation). The resulting VLM is then fine-tuned on individual downstream datasets (PaliGemma stage 3). This evaluates the vision encoder's representational quality in the most practically relevant use case: serving as the visual backbone for a multimodal LLM.

Aggregate headline (Figure 4, "Average" bar): SigLIP 2 L/16 256px achieves the highest average score across all 35+ benchmarks compared to SigLIP L/16 256px, AIMv2 L/14 224px, SigLIP So400m/14 224px, SigLIP 2 So400m/14 224px, SigLIP So400m/14 384px, and SigLIP 2 So400m/14 384px. At 256 tokens (L/16 256px, AIMv2 L/14 224px, So400m/14 224px), SigLIP 2 L/16 256px scores approximately 85 on the average metric, compared to SigLIP L/16 at roughly 80 and AIMv2 at roughly 79. The So400m models score higher in absolute terms due to larger capacity, with SigLIP 2 So400m/14 384px reaching approximately 90 (the highest overall).

Where SigLIP 2 excels in VLM transfer (Figure 4, Table 6): The largest gains over SigLIP v1 appear on tasks requiring reading, localization, and fine-grained visual understanding:

• OCR-VQA: SigLIP 2 L/16 256px scores 72.5 vs. SigLIP L/16 256px at 70.6. At So400m/14 384px, the gap widens to 75.2 vs. 74.4—a more modest improvement.
• TextVQA: SigLIP 2 L/16 256px achieves 57.3 vs. SigLIP L/16 at 51.9. The 384px So400m comparison is 74.0 vs. 69.7—a dramatic 4.3 point gain, representing roughly a 6% relative improvement.
• DocVQA: SigLIP 2 L/16 256px scores 35.4 vs. SigLIP L/16 at 33.0. At 384px, the gap explodes: SigLIP 2 So400m reaches 65.9 vs. SigLIP So400m at 62.7, and the 256px comparison shows an even larger relative gap (35.9 vs. 34.3 for the So400m at 224px, but note that at higher resolutions the absolute gains are more pronounced—the L/16 comparison at 256px shows +2.4 points, a 7% relative improvement).
• ST-VQA: SigLIP 2 L/16 256px reaches 59.8 vs. SigLIP L/16 at 54.0—a 10.7% relative improvement. At So400m 384px: 77.3 vs. 75.0.
• RefCOCO variants: Across all 8 RefCOCO/RefCOCO+/RefCOCOg splits, SigLIP 2 shows consistent improvements of 2-5 points at the L/16 256px level (e.g., RefCOCO val: 72.4 vs. 68.7).

Tasks with smaller or negligible gains (Figure 4, Table 6): On standard VQA (VQAv2 minival) and knowledge-intensive tasks (ScienceQA, AOKVQA), the improvements are modest: VQAv2 at L/16 256px is 82.1 for SigLIP 2 vs. 81.5 for SigLIP. ScienceQA is essentially identical (96.2 vs. 96.1). This pattern makes sense: these tasks depend more on the LLM's reasoning and knowledge than on the vision encoder's ability to produce spatially-structured features, so the LocCa and self-supervised improvements to the vision encoder matter less.

Comparison with AIMv2 (Figure 4, Table 6, first three columns): AIMv2 L/14 224px scores 73.2 on AI2D, 62.3 on AOKVQA-DA, and 78.4 on AOKVQA-MC, compared to SigLIP 2 L/16 256px at 75.9, 61.7, and 77.6 respectively—SigLIP 2 leads on AI2D, trails on AOKVQA. On COCO captioning, SigLIP 2 scores 141.3 vs. AIMv2's 139.9. On TextCaps, the gap is dramatic: SigLIP 2 achieves 126.1 vs. AIMv2's 116.8—an 8% relative improvement, highlighting SigLIP 2's advantage on text-heavy tasks. On DocVQA, SigLIP 2 scores 35.4 vs. AIMv2's 32.3. Overall, SigLIP 2 L/16 256px outperforms AIMv2 L/14 224px on the majority of benchmarks, with the largest advantages on OCR and text-reading tasks.

Resolution scaling in VLM transfer (Figure 4, last four columns): Scaling the vision encoder from 224px to 384px for the So400m/14 model produces large gains on text-heavy tasks: DocVQA improves from 35.9 (SigLIP 2 at 224px) to 65.9 (384px), TextVQA from 59.4 to 74.0, ST-VQA from 60.1 to 77.3. The improvement is much smaller on general VQA (VQAv2: 82.8 to 85.2) and image captioning (COCOcap: 142.7 to 143.8). This suggests that for VLM applications involving document understanding or OCR, higher resolution is essential, while for natural image understanding, 256px may be sufficient.

Dense Prediction Tasks (Segmentation, Depth, Normals)

Probing protocol (Section 3.3.1): Frozen vision encoder representations are evaluated by training lightweight probes on top—either a linear layer or a DPT decoder—for semantic segmentation (PASCAL, ADE20k), monocular depth estimation (NYUv2, NAVI), and surface normal estimation (NYUv2, NAVI). The protocol follows TIPS (Maninis et al., 2025) exactly, with one modification: where the original concatenates the CLS token to each patch feature, SigLIP 2 concatenates the MAP head output instead (since SigLIP uses MAP pooling rather than a CLS token).

Segmentation (Table 2): SigLIP 2 So400m/14 224px achieves 77.1% mIoU on PASCAL, compared to SigLIP So400m/14 at 72.0%—a 5.1 percentage point gain (7.1% relative improvement). On ADE20k, the improvement is from 37.6% to 41.8% (4.2 point gain). At 384px, SigLIP 2 reaches 78.1% on PASCAL and 45.4% on ADE20k, compared to SigLIP v1's 73.8% and 40.8%. The gap to other open models is substantial: CLIP L/14 224px scores 74.5% PASCAL mIoU and 39.0% ADE20k; OpenCLIP G/14 (a much larger model) scores only 71.4% PASCAL and 39.3% ADE20k.

Depth estimation (Table 2): On NYUv2, SigLIP 2 So400m/14 224px achieves 0.493 RMSE vs. SigLIP v1's 0.576—lower is better, and this represents a 14.4% relative improvement in prediction error. At 384px, the improvement is similar: 0.466 vs. 0.563 (17.2% relative improvement). On NAVI depth, the pattern holds: 0.067 vs. 0.083 at 224px, and 0.064 vs. 0.069 at 384px. CLIP L/14 scores 0.553 on NYUv2 and OpenCLIP G/14 scores 0.541—both substantially worse.

Surface normal estimation (Table 2): The improvements are more modest but consistent. On NYUv2 normals, SigLIP 2 So400m/14 224px achieves 24.9° angular RMSE vs. SigLIP v1's 25.9°, and at 384px: 23.0° vs. 24.1°. On NAVI normals, the improvement is marginal: 25.4° vs. 26.0° at 224px, and 25.0° vs. 25.4° at 384px. This is the task where the self-supervised losses provide the smallest benefit, possibly because surface normal estimation requires very fine-grained geometry that the contrastive + masked prediction objectives don't fully capture.

Comparison with baselines across all dense tasks (Table 2): The consistent pattern is SigLIP 2 So400m/14 > SigLIP So400m/14 > CLIP L/14 ≈ OpenCLIP G/14. The improvement over CLIP is particularly large for depth (0.493 vs. 0.553, a 10.8% improvement) and segmentation (77.1% vs. 74.5%). This validates the central claim that combining self-supervised losses with contrastive training produces better dense features than either approach alone.

Open-Vocabulary Segmentation

Evaluation framework (Section 3.3.2): The Cat-Seg framework (Cho et al., 2024) is trained on COCO-Stuff-164k with 172 classes and tested on datasets with different vocabularies, evaluating the model's ability to segment novel classes not seen during training.

Results (Table 3): SigLIP 2 L/16 outperforms SigLIP L/16 across all 6 benchmarks: A-847 (14.3% vs. 14.0%), PC-459 (24.1% vs. 23.9%), A-150 (38.8% vs. 37.5%), PC-59 (62.4% vs. 61.6%), VOC-20 (97.0% vs. 96.1%), and VOC-21 (82.3% vs. 81.1%). The improvements are small in absolute terms (0.3-1.3 percentage points) but consistent. More impressively, SigLIP 2 L/16 matches or exceeds the much larger OpenCLIP G/14 model (which has substantially more parameters) on most benchmarks: A-847 (14.3% vs. 13.3%), PC-459 (24.1% vs. 21.4%), A-150 (38.8% vs. 36.2%), PC-59 (62.4% vs. 61.5%), VOC-20 (97.0% vs. 97.1%—OpenCLIP G/14 edges ahead slightly), and VOC-21 (82.3% vs. 81.4%). This demonstrates that the improved dense features from SigLIP 2 transfer to open-vocabulary settings where the model must generalize to unseen categories.

Referring Expression Comprehension

Evaluation protocol (Section 3.4.1): A 6-layer transformer decoder is attached to the frozen, un-pooled vision encoder (trained from scratch on a mix of all RefCOCO variants) and evaluated on RefCOCO, RefCOCO+, and RefCOCOg. This specifically measures the localization information retained in the vision encoder's patch-level features.

Headline results (Table 5): SigLIP 2 L/16 256px achieves 86.04% on RefCOCO val versus SigLIP v1 L/16 256px at 67.33%—an 18.71 percentage point gain (27.8% relative improvement). This is one of the largest single-task improvements in the entire paper and directly validates the LocCa decoder's effectiveness at forcing the vision encoder to preserve spatial information.

Detailed RefCOCO results (Table 5, L/16 256px row): On RefCOCO testA (which typically involves easier, more distinctive objects), SigLIP 2 scores 89.02% vs. SigLIP v1's 72.40% (+16.62 points). On testB (more challenging, less distinctive objects), the gain is even larger: 81.85% vs. 61.21% (+20.64 points). On RefCOCO+ (which excludes spatial relationship language, making it harder), SigLIP 2 scores 77.29%/83.28%/70.16% on val/testA/testB versus SigLIP v1's 59.57%/67.09%/51.08%—gains of roughly 17-19 points across the board. On RefCOCOg (involving longer, more natural language descriptions), SigLIP 2 achieves 80.11%/80.78% on val/test versus SigLIP v1's 61.89%/62.90%. The consistency of the improvement across all RefCOCO variants is striking and suggests the LocCa decoder produces a fundamental improvement in localization capability rather than an improvement specific to certain types of referring expressions.

Comparison with Cap and CapPa (Table 5, L/16 256px row): The pure image captioner Cap achieves 60.64% on RefCOCO val, and CapPa (with parallel prediction) reaches 64.17%. SigLIP 2's 86.04% dramatically exceeds both—by over 20 percentage points—demonstrating that while captioning alone provides some localization signal (Cap/P upgrade over CLIP's 65.21%), the additional referring expression and grounded captioning tasks in the LocCa decoder provide a much stronger training signal. This is a key finding: it's not just "having a decoder" that matters, but what you train the decoder to do.

Comparison with LocCa (Table 5, L/16 256px row): LocCa (Wan et al., 2024) achieves 88.34% on RefCOCO val—still 2.3 points ahead of SigLIP 2's 86.04%. The paper hypothesizes that "SigLIP 2 is pretrained on multilingual data. LocCa, on the other hand, is trained on text only from English web sites." This is a plausible explanation: the LocCa model can specialize its limited capacity entirely on English-language localization, while SigLIP 2 must distribute capacity across 109 languages, likely leaving some localization performance on the table. The paper also notes that they "expect significant improvements when using the decoder from pretraining" (i.e., the LocCa decoder that was trained jointly with the vision encoder, rather than training a new decoder from scratch), suggesting the 2.3-point gap could be closed with better decoder initialization.

Scaling with model size and resolution (Table 5, all rows): For SigLIP 2, the referring expression scores scale with both model size and image resolution. Going from B/16 256px to L/16 256px: RefCOCO val improves from 83.76% to 86.04% (+2.28 points). Going from L/16 256px to So400m/16 256px: 86.04% to 86.42% (a much smaller gain, suggesting diminishing returns with model size for this task). Going from L/16 256px to L/16 576px: 86.04% to 87.28% (+1.24 points from higher resolution). Going to the ViT-g 576px: 88.45% on RefCOCO val and 91.53% on testA—the highest reported numbers in the table, though still slightly behind LocCa L/16 256px (which achieves 91.20% on testA with a much smaller model at lower resolution, underscoring the English-only vs. multilingual tradeoff).

Open-Vocabulary Detection

Evaluation setup (Section 3.4.2, Table 4): The OWL-ViT method (Minderer et al., 2022) is applied to adapt SigLIP and SigLIP 2 models for open-vocabulary detection, with evaluation on COCO and LVIS.

Results (Table 4): For B/16, SigLIP 2 achieves 42.8% AP on COCO versus SigLIP v1's 42.2% (+0.6 points). On LVIS, SigLIP 2 scores 34.4% AP versus 33.0% (+1.4 points). The gap on LVIS rare categories (APr) is larger: 32.7% vs. 31.0% (+1.7 points). For the So400m/14 model, the pattern is similar: COCO AP improves from 44.3% to 45.2% (+0.9 points), LVIS AP from 39.5% to 40.5% (+1.0 points), and LVIS APr from 40.9% to 42.3% (+1.4 points). The consistent improvement on LVIS rare categories is notable because these are classes with few training examples, where the vision encoder's generalization capability matters most. The improvement over SigLIP v1 is modest compared to the gains on referring expressions (1-2 points here vs. 18 points for RefCOCO), likely because the OWL-ViT detection head has limited capacity and may not fully exploit the improved patch-level features.

Cultural Diversity and Fairness

Cultural diversity (Figure 5, Table 8 in Appendix C): The paper evaluates zero-shot and 10-shot classification on geographically diverse benchmarks: Dollar Street (household items across income levels), GeoDE (objects across regions), and GLDv2 (landmark recognition across countries).

Key finding on geolocalization (Figure 5, Table 8): SigLIP 2 L/16 256px achieves 10-shot geolocalization accuracy on GeoDE (country) of 34.5%, compared to SigLIP L/16 256px at 22.1%—a 12.4 percentage point gain (56% relative improvement). On GeoDE (region), the improvement is from 36.2% to 44.4% (+8.2 points). These are among the largest relative improvements for any task in the paper and directly validate the multilingual training data's impact on geographically diverse recognition.

Dollar Street and GeoDE object recognition (Figure 5, Table 8): On Dollar Street zero-shot, SigLIP 2 L/16 256px scores 55.2% versus SigLIP L/16 256px at 52.1% (+3.1 points). On GeoDE zero-shot, the improvement is from 93.6% to 94.9% (+1.3 points—a smaller gain because the baseline is already high). The 10-shot improvements are larger: Dollar Street 10-shot improves from 18.8% to 26.8% (+8.0 points), and GeoDE country from 22.1% to 34.5% (+12.4 points). The larger 10-shot gains suggest that the multilingual data provides better feature representations that enable more effective few-shot adaptation—the model can leverage its broader visual-linguistic knowledge when given a small number of labeled examples.

GLDv2 landmark recognition (Table 8): SigLIP 2 L/16 256px achieves 64.5% zero-shot on GLDv2 vs. SigLIP L/16's 56.7%—a 7.8 percentage point gain. This task requires recognizing specific landmarks (e.g., the Eiffel Tower, the Taj Mahal) from images, which depends on both visual recognition and geographic knowledge. The improvement likely comes from the multilingual training exposing the model to a wider range of geographic contexts and landmark names in multiple languages.

Scaling trends for cultural diversity (Table 8): For both SigLIP v1 and SigLIP 2, larger models perform better on cultural diversity metrics. For GeoDE country 10-shot, SigLIP 2 scales from 20.0% (B/16 224px) to 34.5% (L/16 256px) to 38.1% (So400m/14 224px) to 46.6% (g/16 256px) to 52.0% (g/16 384px). The g/16 384px model nearly matches the 10-shot GeoDE accuracy of some supervised models, demonstrating that scale helps with geographic generalization. Interestingly, mSigLIP (So400m/16 256px) scores 33.3% on GeoDE country 10-shot—lower than SigLIP 2 at 39.8%—despite being trained on 100% multilingual data. This suggests that SigLIP 2's 90/10 split, combined with the other recipe improvements, produces better geographic generalization than the purely multilingual training approach.

Representation bias (Figure 6, Table 9 in Appendix C): This metric measures the tendency to associate random objects with a particular gender, following the methodology from Alabdulmohsin et al. (2024). A score of 50% would mean the model shows no gender preference; higher scores indicate bias toward associating objects with one gender.

Key finding: SigLIP 2 L/16 256px has a representation bias of 7.3%, compared to SigLIP L/16 256px at 35.5%—a reduction of 28.2 percentage points. This is a dramatic improvement. At the B/16 224px scale, SigLIP 2 scores 17.2% vs. SigLIP's 36.6%. At So400m/14 224px, SigLIP 2 scores 7.4% vs. SigLIP's 33.3%. The paper attributes this directly to the debiasing techniques from Alabdulmohsin et al. (2024) applied during data preprocessing (Section 2.1): "we further apply the filtering techniques from [2] to mitigate data biases in representation and association with respect to sensitive attributes."

The effect of model scale on bias (Table 9): Within SigLIP 2, larger models exhibit lower representation bias: B/16 256px = 19.4%, L/16 256px = 7.3%, So400m/16 256px = 7.2%, g/16 256px = 7.9% (a slight increase, but still much lower than any SigLIP v1 model). This confirms the finding from Alabdulmohsin et al. (2024) that "larger models tend to exhibit less representation bias than smaller models" when trained with debiasing techniques.

Performance disparity by income (Table 9): The maximum difference in zero-shot accuracy on Dollar Street when disaggregating by income level (a measure of how much the model's performance varies across socioeconomic groups) is relatively stable across SigLIP v1 and SigLIP 2. For L/16 256px, SigLIP v1 has a disparity of 32.0% vs. SigLIP 2 at 31.1%—a modest improvement. For So400m/14 224px: 30.5% (SigLIP v1) vs. 29.7% (SigLIP 2). The gains in fairness are concentrated in representation bias (gender association), not in performance equity across income levels. The paper is transparent about this: "in this context we only observe very minor benefits, or no benefits when comparing SigLIP and SigLIP 2 models of matching size and resolution" (Section 3.5).

---

Ablation Studies and Robustness Checks

The paper does not contain traditional ablation studies—there is no experiment that removes the LocCa decoder and measures the impact, no experiment that adds self-supervised losses from the start versus at 80%, and no experiment that varies the language mixture ratio and observes the effect on multilingual vs. English performance. This is a significant gap: the paper's central claim is that these techniques compose without interference, but it never demonstrates that each component individually contributes to the final performance through controlled removal experiments. Instead, the evidence is provided indirectly through comparisons with baseline models that lack various components:

• SigLIP v1 → SigLIP 2 (adding LocCa, SILC/TIPS, multilingual data, debiasing): The full effect of the combined recipe versus the original recipe. Table 1, Table 2, and Table 5 provide this comparison across all tasks, showing consistent improvements. However, this conflates multiple changes and cannot attribute improvements to specific components.
• SigLIP 2 vs. mSigLIP (the effect of the 90/10 language mixture vs. 100% multilingual): Table 1 and Figure 2 show that SigLIP 2 nearly matches mSigLIP on multilingual retrieval while dramatically exceeding it on English benchmarks. This is an implicit ablation of the language mixture ratio.
• NaFlex vs. standard SigLIP 2 (the effect of self-supervised losses and data curation): Figure 3 shows that NaFlex (which omits Sections 2.3 and 2.5) underperforms standard SigLIP 2 on natural image benchmarks but wins on text-heavy benchmarks, illustrating the value of these components for different task families.

LocCa decoder contribution: The only indirect ablation of the LocCa decoder comes from Table 5. Comparing SigLIP 2 L/16 256px (86.04% RefCOCO val) to SigLIP L/16 256px (67.33%) shows a massive improvement, but this conflates the decoder-based pretraining with the other recipe changes (multilingual data, debiasing). A cleaner comparison would be SigLIP 2 L/16 trained without the LocCa decoder versus with it, but this experiment is not reported.

Self-supervised loss contribution: The indirect evidence comes from comparing dense prediction results in Table 2. SigLIP 2 So400m/14 224px achieves 77.1% PASCAL mIoU vs. SigLIP v1's 72.0%, but this again conflates multiple changes. Additionally, the NaFlex variant (which omits self-supervised losses) underperforms the standard variant on natural images (Figure 3), which provides weak evidence that self-supervised losses benefit these tasks—but NaFlex also omits active data curation and uses a different training schedule, making the comparison multiply confounded.

Active data curation contribution: The B/16 and B/32 models receive the ACID stage (Section 2.5) while the L/16, So400m, and g models do not. The B/16 at 224px shows a 2.0 percentage point improvement over SigLIP v1 B/16 (78.2% vs. 76.2%), while the L/16 at 256px also shows a 2.0 point improvement (82.5% vs. 80.5%). This is a weak comparison because L/16 starts from a higher baseline and might show different scaling behavior, but the similar absolute improvement across scales suggests the active data curation may not be the dominant factor driving the gains—the other recipe changes (LocCa, multilingual data) likely contribute substantially to both B/16 and L/16 improvements.

MAP head vs. CLS token for dense probing: Section 3.3.1 notes the single necessary change to the TIPS evaluation protocol: concatenating the MAP head output instead of the CLS token to each patch feature vector. This is presented as an implementation detail, but it represents an implicit architectural ablation. The fact that SigLIP 2 achieves strong dense prediction results with this modified protocol (Table 2) suggests that the MAP head produces a useful global representation that complements patch-level features for dense tasks—not something the paper discusses explicitly, but a robustness check on the protocol nonetheless.

Resolution adaptation strategy (95% checkpoint vs. final checkpoint fine-tuning): Section 2.4.1 states that "the common strategy of fine-tuning the final checkpoint with smaller learning rate and without weight decay did not lead to good results across all sizes and resolutions." This is presented as a methodological finding rather than a formal ablation—the paper tried the standard approach, found it insufficient, and switched to the 95% continuation strategy. No quantitative comparison is provided, but the claim suggests that the resolution adaptation benefit (Table 1 shows consistent improvement from 224px → 256px → 384px → 512px) depends on this specific adaptation approach.

Negative result: ReST^EM-style revision training for vision encoders (Appendix K): The paper does not report negative results for SigLIP 2 components in the main text. However, the decision not to include certain combinations is itself informative: NaFlex omits self-supervised losses because "implementation and computation complexity" would be unmanageable, and the self-supervised losses are added at 80% rather than from the start (the alternative is not tested but implicitly rejected). These are design decisions rather than formal negative results, but they reflect choices made to avoid problems observed in preliminary experiments.

---

Critical Assessment

The paper's central claims, evaluated against the presented experiments:

Claim 1: SigLIP 2 outperforms SigLIP v1 "at all model scales in core capabilities." The experiments in Table 1 (zero-shot classification, retrieval) and Table 2 (dense prediction) provide strong support: at every model size (B/16, B/32, L/16, So400m/14, g/16) and every resolution, SigLIP 2 scores higher than SigLIP v1 on every metric reported. The improvements range from modest (ImageNet-1k: +1.9-2.6 points) to dramatic (RefCOCO val: +18.7 points; Crossmodal-3600 T→I: +15.6 points). The g/16 model, which has no SigLIP v1 counterpart, is excluded from this comparison. The claim holds unconditionally for the SigLIP v1 → SigLIP 2 comparisons that are reported.

Claim 2: The improvements extend to "localization and dense prediction tasks." This is strongly supported by Table 5 (RefCOCO improvements of 17-21 points across variants and model sizes), Table 2 (PASCAL mIoU +5.1 points, NYUv2 depth RMSE -0.083), and Table 4 (open-vocabulary detection +0.6-1.7 points). The gains are larger for localization (referring expressions) than for dense prediction, and larger for depth than for normals—the improvements are not uniform but are consistently positive.

Claim 3: The model provides "strong multilingual vision-language encoders" that work with "a single model." Figure 2 and Table 1 (Crossmodal-3600 columns) support this: SigLIP 2 achieves 46.5% average T→I recall across 36 languages (L/16 256px) versus SigLIP v1's 30.9%—a massive improvement—while simultaneously improving English benchmarks. The claim that SigLIP 2 "almost matches" mSigLIP on multilingual needs qualification: SigLIP 2 L/16 achieves 46.5% T→I vs. mSigLIP's 50.0%—a 3.5 point gap that increases for low-resource languages. "Almost matches" is a fair characterization but overstates the parity; "substantially closes the gap" would be more precise.

Claim 4: The NaFlex variant "preserves the input's native aspect ratio" and "supports multiple resolutions." Figure 3 and Table 7 confirm that NaFlex processes images at native aspect ratios and achieves competitive or superior performance on text-heavy benchmarks. However, the claim that NaFlex is a viable single-checkpoint replacement for multiple fixed-resolution checkpoints is only partially supported: for the B/16 model, the standard variant consistently outperforms NaFlex on natural image benchmarks (ImageNet-1k: 78.2% vs. 76.2% at comparable resolutions), and the paper acknowledges this gap by attributing it to NaFlex omitting self-supervised losses and distillation. For the So400m model, the two variants are nearly identical, suggesting that NaFlex loses less at larger scale.

Claim 5: The recipe produces "strong small models" through distillation via active data curation. The B/16 and B/32 models do outperform their SigLIP v1 counterparts (Table 1), and the B/32 model substantially outperforms prior B/32 baselines (74.0% vs. OpenCLIP's 72.8%). However, there is no ablation that isolates the ACID contribution—the B/16 improvement could come from the other recipe changes (LocCa, multilingual data, debiasing) rather than the distillation stage. The paper's evidence for ACID's effectiveness is primarily citation-based (referencing Udandarao et al., 2024) rather than experimentally demonstrated within the SigLIP 2 context.

Genuine weaknesses in experimental design:

• No component ablations. The paper's title and introduction position it as "building on SigLIP with several prior, independently developed techniques into a unified recipe," but it never demonstrates—through controlled removal experiments—that each technique contributes independently. Did the LocCa decoder matter, or would the self-supervised losses alone have achieved similar localization improvements? Did the 90/10 language mixture matter, or would a smaller non-English fraction have sufficed? Without ablations, the paper is a single point in the design space: "this combination works." It provides no information about why it works, which components are essential versus nice-to-have, or how robust the results are to recipe variations.

• Single data source. All models are trained on WebLI, a proprietary Google dataset. The results may be specific to WebLI's distribution of images and alt-texts, its language coverage, and its noise characteristics. Reproducing the recipe on other corpora (LAION, DataComp, COYO) may yield different results, and the absence of any cross-dataset validation limits the generalizability of the findings.

• No statistical significance reporting. The paper reports point estimates on fixed test sets with no confidence intervals, standard deviations, or significance testing. For the subgroup analyses in the fairness section (Dollar Street by income, GeoDE by region), sample sizes within subgroups may be small enough that chance variation could explain some of the reported differences. The lack of statistical rigor makes it difficult to assess whether small improvements (e.g., 0.3-1.3 point gains on open-vocabulary segmentation in Table 3) are reliable or within noise.

• No comparison with CLIP-style models that incorporate some but not all improvements. The baseline set is selected to represent the state of prior open releases, but there is no comparison with a model that includes, say, self-supervised losses but no LocCa decoder, or multilingual data but no debiasing. These intermediate baselines would help attribute improvements to specific techniques. The comparison with LocCa (Table 5) is the closest the paper comes to this, and it reveals that SigLIP 2 trails a model trained only with decoder-based losses on localization—an informative finding that would benefit from similar side-by-sides for other components.

• VLM transfer experiments use only Gemma 2 2B. All VLM experiments (Section 3.2, Figure 4) pair the vision encoder with a single LLM architecture at a single scale. The finding that SigLIP 2 outperforms SigLIP v1 as a vision encoder for VLMs is demonstrated for this specific LLM, but whether the benefit transfers to larger LLMs (Gemma 7B, Llama-70B), different architectures (non-Gemma), or different multimodal training recipes is untested. Prior work (Tong et al., 2024, Cambrian-1) has shown that optimal vision encoder choice can depend on LLM scale—larger LLMs may benefit more from higher-capacity vision encoders, or the benefit of improved vision features may saturate.

• No analysis of computational cost of the unified recipe versus simpler recipes. The paper never reports the training FLOPs, wall-clock time, or GPU/TPU-hours for the SigLIP 2 recipe versus SigLIP v1. The addition of the LocCa decoder (three forward passes per example), the self-supervised losses (8+1 additional augmented views), and the ACID stage (teacher scoring of super-batches) all increase training cost. Without cost reporting, it's unclear whether the improvements are "worth it" in a compute-matched sense, or whether training a larger SigLIP v1 model would have achieved similar results more cheaply. This is the same criticism the paper leveled at prior work in its motivation—that techniques were validated in isolation without unified analysis—and it applies to SigLIP 2 as well.

• The NaFlex variant's omission of self-supervised losses is a significant limitation for the claimed "single model" benefit. Figure 3 shows that NaFlex B/16 underperforms standard B/16 on natural image benchmarks, and the paper acknowledges this is because NaFlex omits the self-supervised and distillation stages. This means that the "single checkpoint" convenience of NaFlex comes at a real accuracy cost—especially for smaller models. A user who needs maximum performance on natural images must still deploy separate checkpoints, undermining the practical value proposition of NaFlex for that use case.

Missing experiments that would have strengthened the paper:

• Ablation of the 80% staging decision for self-supervised losses: what happens if they are added from the start? At 50%? At 95%?
• Ablation of the LocCa decoder: train with sigmoid loss only (matching SigLIP v1's objective) but on the same multilingual, debiased data to isolate the data effect from the decoder effect.
• Ablation of the self-supervised loss weighting (the 0.25/0.5/1.0/0.5 per-model-size factors): what happens with uniform weighting?
• Ablation of the 90/10 language mixture: results at 50/50, 75/25, 95/5 to show the sensitivity of the English-vs-multilingual tradeoff.
• A compute-matched comparison: given the total FLOPs of the SigLIP 2 recipe, what ImageNet accuracy would a SigLIP v1 model achieve if trained for the equivalent number of steps, or if scaled up in parameters?
• Confidence intervals on all major benchmark results to assess the reliability of small improvements.
• Generalization to non-WebLI data: zero-shot evaluation of SigLIP 2 trained on WebLI versus models trained on LAION/DataComp, or fine-tuning SigLIP 2 on non-WebLI data to test transfer.

Conditional nature of the claims:

• The improvements hold for tasks where the base SigLIP v1 model has non-trivial performance. There is no evidence that SigLIP 2 unlocks fundamentally new capabilities—it improves existing ones. The largest relative gains are on tasks where SigLIP v1 already works (RefCOCO, depth estimation), and the smallest gains are on tasks near ceiling (VQAv2, ScienceQA) or floor (the hardest categories in LVIS).
• The multilingual improvements are concentrated in higher-resource languages. Figure 2 shows that SigLIP 2 nearly matches mSigLIP for French, German, Spanish, Italian, and Chinese, but trails substantially for Maori, Telugu, and Cusco Quechua. The claim of "strong multilingual" performance should be qualified with "particularly for higher-resource languages"—a point the paper acknowledges implicitly by showing the per-language breakdown but does not discuss in the main text.
• The benefit of NaFlex is task-dependent. Figure 3 shows NaFlex excels on text-heavy benchmarks (HierText, SciCap, Screen2Words) but underperforms on natural image benchmarks, especially for smaller models. The paper's framing of NaFlex as a positive contribution is accurate but requires the caveat that it involves a tradeoff unless the deployment use case is document/UI-focused.
• The fairness improvements are concentrated in representation bias, not performance equity. Table 9 shows dramatic reductions in gender association bias (35.5% → 7.3% for L/16) but minimal changes in income-level performance disparity (32.0% → 31.1%). The debiasing techniques address one specific type of bias effectively, but the model still exhibits substantial performance gaps across socioeconomic groups that the training recipe does not resolve.

Overall assessment: The experiments provide strong evidence that SigLIP 2's unified recipe produces models that outperform SigLIP v1 across a diverse set of tasks—the improvements are consistent in direction and substantial in magnitude on several important benchmarks (RefCOCO, Crossmodal-3600, depth estimation). However, the absence of component ablations means the paper cannot explain why the recipe works—which techniques are load-bearing and which are incidental—and this limits both the scientific contribution and the practical guidance for practitioners who want to adopt parts of the recipe without reproducing it in full. The paper is best understood as a demonstration of existence: a model that combines these techniques exists and performs well. It is not a systematic analysis of the interactions between the techniques or a guide to which techniques are most important—those questions remain open.

6. Limitations and Trade-offs

The Difficulty Estimation Cost Is Excluded from Headline Efficiency Gains

The assumption or constraint: The entire compute-optimal allocation framework from the reference paper depends on estimating each prompt's difficulty before deciding how to spend the inference budget. The method used—generating 2048 samples per question and averaging either ground-truth correctness (oracle) or PRM scores (predicted)—is extraordinarily expensive. The paper acknowledges this explicitly in Section 3.2:

"estimating difficulty in this way still incurs additional computation cost during inference... our experiments do not account for this cost largely for simplicity"

The consequence: The reported 4× efficiency gains over best-of-N are computed after difficulty is known, without amortizing the cost of estimating it. In a realistic deployment, the total cost would be difficulty estimation + strategy execution, and the former—requiring 2048 complete generations per question—could easily dwarf the latter. For context, 2048 samples per question already exceeds the largest test-time compute budgets studied (256–512 generations), meaning the difficulty estimation step alone consumes more compute than the entire problem-solving phase. This makes the 4× figure an upper bound on achievable efficiency in a real system, not a realized deployment gain. The gap between this upper bound and practical reality is unknown and potentially very large.

What evidence exists in the paper: The paper provides no experiment measuring the total cost including difficulty estimation, no ablation showing how performance degrades with fewer difficulty estimation samples, and no alternative difficulty estimation method with lower cost. The difficulty estimation cost is mentioned in Section 3.2 as a limitation and left as future work.

Mitigation status: Not addressed. The paper suggests future work on "pretraining or fine-tuning models to directly predict difficulty of a question" (Section 8), but no such model is developed or evaluated. An adaptive scheme—starting with a few samples to estimate difficulty, then allocating the remaining budget—is mentioned as a possibility but not explored. Until the difficulty estimation cost is reduced by orders of magnitude or absorbed into the problem-solving process, the compute-optimal framework remains a proof of concept rather than a practical deployment strategy.

---

Hard Problems Remain Essentially Unsolved

The assumption or constraint: The compute-optimal framework assumes the base model has a non-trivial probability of producing a correct answer—if the base model's pass@1 on a problem class is near zero, no amount of test-time search or revision can help because there are no correct solutions in the proposal distribution to find or refine. The paper is transparent about this:

"on the hardest questions (bin 5), no method makes meaningful progress regardless of budget" (Section 5.3)

The consequence: Test-time compute scaling has a hard ceiling determined by the base model's capabilities. For problems genuinely outside the model's training distribution or reasoning capacity, the approach offers zero benefit regardless of compute budget. In the FLOPs-matched comparison (Section 7, Figure 9), difficulty bin 5 accuracy hovers at 1–5% for all methods and all budgets—essentially at chance. This means the approach cannot substitute for pretraining on fundamentally novel or out-of-distribution tasks. A practitioner facing a deployment where even a small fraction of queries fall into this "too hard" category must either accept near-zero accuracy on those queries or invest in larger-scale pretraining, because test-time compute provides no recourse.

What evidence exists in the paper: Figure 3 (right), Figure 7 (right), and Figure 9 all show difficulty bin 5 accuracy flat near 0–5% across all methods (search, revisions, compute-optimal combinations) and all compute budgets. The FLOPs-matched comparison (Figure 9, Table in Section 7) shows that on hard problems, the ~14× larger pretrained model substantially outperforms test-time compute scaling—e.g., a -52.9% relative disadvantage for PRM search on hard questions at R ≫ 1.

Mitigation status: Not addressed and likely fundamental. The paper acknowledges this limitation explicitly (Section 7 takeaway box) but offers no solution beyond scaling pretraining. The boundary between "solvable with test-time compute" and "requires more pretraining" is a property of the base model's training distribution, and no inference-time technique can bridge it. This is not a flaw of the paper's approach but a fundamental constraint on what test-time compute can achieve.

---

The ~14× Larger Model Baseline Is Not Compute-Optimally Trained

The assumption or constraint: The FLOPs-matched comparison in Section 7 scales model parameters by ~14× while holding training data fixed, following the LLaMA paradigm (Touvron et al., 2023) rather than the Chinchilla-optimal approach of scaling both parameters and data equally (Hoffmann et al., 2022). The paper acknowledges this departure:

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally to future work" (Section 7)

Additionally, the larger model uses only greedy decoding—no majority voting, no best-of-N, no search—while the smaller model with test-time compute gets the full benefit of the compute-optimal strategy.

The consequence: Both choices make the pretraining baseline weaker than it could be. A Chinchilla-optimal model trained with the same total FLOPs would likely outperform a parameter-only-scaled model, and giving the larger model even a modest test-time compute budget (say, best-of-8) would create a much stronger comparison. The reported advantages of test-time compute over pretraining—e.g., +27.8% relative improvement on easy-medium questions at R ≪ 1 for revisions—may shrink or reverse against a properly compute-optimal larger model with some test-time augmentation. The paper's central claim about the pretraining-inference tradeoff is therefore conditional on the specific, suboptimal pretraining recipe used for the larger baseline, and the magnitude of this conditionality is unknown.

What evidence exists in the paper: No experiment in the paper tests a stronger pretraining baseline. The paper provides the raw numbers (Figure 9) that allow a reader to assess the gap, but there is no sensitivity analysis varying the pretraining scaling strategy or giving the larger model test-time compute. The bar charts in Figure 1 show the relative advantage of test-time compute but inherit the baseline's weaknesses.

Mitigation status: The paper explicitly flags this as future work but does not attempt to bound the effect. A reader cannot determine from the presented experiments whether test-time compute would still be preferable to compute-optimal pretraining, or whether the reported advantages are artifacts of the specific baseline choice. For practitioners making resource allocation decisions, this is a critical uncertainty: the paper's recommendation to invest in test-time compute over larger pretraining only holds if your alternative is a parameter-scaled, greedily-decoded model—not if you can train compute-optimally and deploy with modest inference-time augmentation.

---

Single Benchmark and Single Model Family

The assumption or constraint: All experiments use the MATH benchmark (500 test questions) with PaLM 2-S* as the base model. The authors state they "believe this model is representative of the capabilities of many contemporary LLMs" (Section 4), but this claim is unverified on other model families (GPT, LLaMA, Gemini, Claude) or other reasoning benchmarks (GSM8K, MMLU reasoning subsets, coding benchmarks like HumanEval, or scientific reasoning tasks).

The consequence: Several aspects of the findings could be model-specific or benchmark-specific in ways that affect generalizability. The PRM's quality and over-optimization behavior depend on PaLM 2-S*'s specific output distribution and calibration properties; a model with different error patterns might produce different difficulty-dependent scaling curves. The revision model's ability to learn from edit-distance-paired incorrect-to-correct trajectories depends on PaLM 2-S*'s in-context learning and fine-tuning dynamics. The MATH benchmark consists of competition-level math problems requiring multi-step symbolic reasoning; the paper's finding that beam search hurts easy problems but helps medium problems might not hold in other reasoning domains (code generation, where structural constraints differ; commonsense reasoning, where "steps" are less well-defined; factual QA, where correctness depends on knowledge retrieval rather than logical deduction). More broadly, the 500-question test set, split into five difficulty quintiles of ~100 questions each and further split by two-fold cross-validation, means the compute-optimal policy is selected based on ~50 questions per fold per bin—a small sample that could introduce variance in strategy selection that the paper does not quantify.

What evidence exists in the paper: No experiment evaluates on any benchmark other than MATH or any model family other than PaLM 2. The paper provides no replication on GSM8K (which has different problem characteristics), no testing with GPT or LLaMA models as the base, and no analysis of how the optimal policies differ across benchmarks. The compute-optimal strategy lookup tables are specific to PaLM 2-S* on MATH and may not transfer.

Mitigation status: Not addressed. The paper acknowledges the single-model limitation in passing but treats it as acceptable because PaLM 2-S* is "representative"—a claim that cannot be evaluated without cross-model replication. The single-benchmark limitation is not discussed. A practitioner using a different model family (e.g., LLaMA for open-weight deployment) or a different task domain (e.g., code generation) cannot assume the paper's specific difficulty thresholds, optimal search algorithms, or sequential-to-parallel ratios will transfer—the entire compute-optimal policy would need to be re-derived.

---

The Revision Model Has a 38% Failure Rate on Correct Answers

The assumption or constraint: The revision model is trained exclusively on sequences where all in-context answers are incorrect followed by a correct target. During training, the model never sees an example where the current answer is already correct and should be preserved. As a result, at test time, when the model produces a correct answer during a revision chain, it has no training signal for what to do and often "revises" a correct answer into an incorrect one. The paper reports:

"approximately 38% of correct answers get converted back to incorrect ones" (Section 6.1)

The consequence: The revision model cannot be used naïvely to produce a single chain of revisions where the final answer is taken as the output—38% of the time, the model actively degrades a correct answer it has already produced. The paper mitigates this with per-chain answer selection (majority voting or verifier-based selection across all revisions in the chain, picking the best answer from any point rather than always taking the last revision), but this is a patch, not a fix. The mitigation adds computational overhead (the verifier must score every step in the chain) and introduces its own errors (the verifier may select an incorrect answer that scores higher than the correct one). More fundamentally, the 38% reversion rate means the revision chain cannot be relied upon to monotonically improve—progress is fragile, and the model can undo its own successes.

What evidence exists in the paper: The 38% figure is reported in Section 6.1, derived from analyzing revision trajectories. Figure 6 (left) shows that pass@1 at each step climbs from ~18% to ~24-25% but visibly fluctuates rather than monotonically increasing, consistent with occasional reversions. The paper also reports in Appendix K that a ReST^EM-trained revision model suffers even worse degradation with sequential revisions, suggesting the reversion problem is sensitive to training methodology.

Mitigation status: Partially addressed. The paper uses within-chain selection to mitigate the reversion problem, but this adds cost and does not eliminate the underlying issue. The paper does not explore training the revision model to recognize when no revision is needed (e.g., by including correct-to-correct trajectories in the training data, or by training a "stop revising" classifier). The ReST^EM negative result (Appendix K, Figure 16) indicates that naïve attempts to further optimize the revision model can make the problem worse, suggesting the current training recipe is brittle with respect to data distribution. For practitioners, this means the revision model requires careful handling (always use within-chain selection, never trust the final output of a chain) and the revision approach is unlikely to benefit from straightforward extensions like longer chains or RL-based optimization without addressing the reversion problem directly.

---

No Ablation of Individual Components in the Training Recipe

The assumption or constraint: The paper presents SigLIP 2 as a unified recipe combining multiple independently-developed techniques—LocCa decoder-based pretraining, SILC/TIPS self-supervised losses, multilingual data mixture, debiasing filters, and ACID-based active data curation for small models. However, the paper provides no component ablation experiments: there is no experiment that removes the LocCa decoder and measures the impact, no experiment that adds self-supervised losses from the start rather than at 80% of training, no experiment that varies the 90/10 English/non-English language mixture, and no experiment that isolates the ACID contribution for B/16 and B/32 models.

The consequence: The paper cannot explain why SigLIP 2 works—which techniques are load-bearing and which are incidental. All comparisons are against external baselines (SigLIP v1, OpenCLIP, DFN, etc.) that differ in multiple ways simultaneously. When SigLIP 2 L/16 improves RefCOCO val from 67.33% (SigLIP v1) to 86.04% (Section 3.4.1, Table 5), it is impossible to determine how much of the 18.7 percentage point gain comes from the LocCa decoder versus the multilingual data versus the debiasing filters versus interaction effects between these components. Similarly, the B/16 and B/32 gains over SigLIP v1 could come from any combination of the shared recipe improvements and the ACID distillation stage. A practitioner who wants to adopt only part of the recipe—e.g., adding the LocCa decoder but not the self-supervised losses because the deployment task doesn't require dense features—has no guidance from the paper about what performance to expect.

What evidence exists in the paper: No ablation experiments exist. The closest the paper comes is: (1) the NaFlex variant (Figure 3, Table 7), which omits self-supervised losses and ACID distillation and shows reduced performance on natural image benchmarks compared to the standard variant—but this comparison is confounded by the different training schedule and aspect ratio processing; (2) the LocCa comparison in Table 5, where LocCa (trained with only decoder-based losses, English-only) outperforms SigLIP 2 on RefCOCO—but this is a comparison against a different model with a different training objective, not an ablation of the LocCa component within SigLIP 2.

Mitigation status: Not addressed. The paper's framing as "combining independently-developed techniques into a unified recipe" implies that the combination itself is the contribution, but without ablations, the contribution is a single point in the design space rather than an analysis of the space. The paper provides no information about which techniques are essential, which are optional, which interact synergistically, and which might even conflict without the specific staging choices (80% self-supervised addition, separate augmentation pipelines). This limits both scientific understanding and practical adoption—a team that wants to adapt SigLIP 2's recipe to their own data, model architecture, or compute budget must either reproduce the entire recipe exactly or conduct their own ablation studies from scratch.

7. Implications and Future Directions

How This Work Changes the Landscape

SigLIP 2 represents an incremental but practically significant reframing of how the field should approach vision-language encoder development: it demonstrates that the fragmentation of improvements across separate research threads—decoder-based pretraining for localization, self-supervised losses for dense features, multilingual data balancing for language coverage, data curation for small-model efficiency—is an artifact of research silos, not a fundamental constraint. The paper does not introduce new loss functions or architectures; its contribution is the compositional proof that these techniques address orthogonal deficiencies in standard contrastive training and can coexist without the destructive interference that prior open releases implicitly assumed by avoiding such combinations.

This matters for three reasons. First, it shifts the default expectation for future open-weight model releases. Prior to SigLIP 2, the major open releases (OpenCLIP, MetaCLIP, EVA-CLIP, DFN, SigLIP v1) all stayed close to the original CLIP formula, improving mainly through data quality. SigLIP 2 sets a new baseline: a model that only does contrastive alignment, without decoder-based losses, self-supervised objectives, or multilingual support, is now visibly incomplete. The paper's consistent Pareto improvement across localization (RefCOCO val: 67.33% → 86.04% for L/16 256px, Table 5), dense prediction (PASCAL mIoU: 72.0% → 77.1% for So400m/14 224px, Table 2), multilingual retrieval (Crossmodal-3600 T→I: 30.9% → 46.5% for L/16 256px, Table 1), and English classification (ImageNet-1k: 80.5% → 82.5% for L/16 256px, Table 1)—all in a single model—means future releases will be measured against this breadth, not just classification and retrieval.

Second, the paper resolves a latent tension in prior work about whether self-supervised and multimodal objectives can coexist. The SILC and TIPS papers demonstrated that adding self-distillation and masked prediction to contrastive training improved dense features, but they were research prototypes, not released models. The broader community could reasonably wonder whether those gains came at the expense of the global alignment quality that practitioners actually deploy for. SigLIP 2's results—where global metrics (zero-shot classification, retrieval) simultaneously improve alongside dense and localization metrics—show that the tension is resolvable with proper staging (self-supervised losses introduced at 80% training, separate augmentation pipelines). This makes the SILC/TIPS line of work immediately actionable for open-source model builders in a way that isolated research papers could not.

Third, the paper's handling of the multilingual-vs-English tradeoff—achieving near-mSigLIP multilingual performance (46.5% vs. 50.0% Crossmodal-3600 T→I for L/16 256px, Figure 2) while exceeding SigLIP v1's English performance—demonstrates that the 90/10 language mixture ratio resolves a problem that the original mSigLIP could not. This is not a theoretical advance but a practical recipe that eliminates the previously assumed "you must choose between English quality and multilingual coverage" dilemma. For the growing number of applications requiring global deployment (VLMs for education, content moderation across languages, international search), this finding changes the default architecture from "deploy separate English and multilingual models" to "deploy a single SigLIP 2 model."

The paper also redirects research attention toward verifier-style diagnostics of aspect ratio distortion through the NaFlex variant. Figure 3 reveals that the standard fixed-resolution training approach systematically disadvantages text-heavy tasks (HierText, SciCap, Screen2Words) at low resolutions, where aspect ratio distortion is most severe—a finding invisible to standard evaluation suites dominated by natural image benchmarks. This is not a new method but a diagnostic contribution: it shows that the field's evaluation practices have implicitly favored models that tolerate distortion, and that the growing importance of document and UI understanding tasks demands a reevaluation of this bias.

However, the paper's impact is limited by what it does not do: it provides no component ablations. The claim that the techniques are complementary rather than overlapping is supported by the end-to-end results but not proven through controlled removal experiments. A skeptical reader could argue that the gains might come primarily from the multilingual data and LocCa decoder, with the self-supervised losses contributing marginally or not at all for most tasks. The paper's value is therefore as a demonstration of a high-water mark—"this combination reaches these numbers"—rather than as a systematic analysis of which components matter and why. Future work that does provide those ablations will determine whether SigLIP 2's recipe is a carefully-balanced house of cards or a robust composition where each piece provides independent value.

Follow-Up Research This Work Enables

Component ablation study of the SigLIP 2 recipe. The most immediate gap the paper leaves is the absence of any controlled removal experiments. A strong follow-up would train SigLIP 2 variants systematically removing one component at a time: (a) sigmoid loss only on the multilingual debiased data (no LocCa, no self-supervised), (b) sigmoid + LocCa (no self-supervised), (c) sigmoid + self-supervised (no LocCa), (d) full recipe but with self-supervised losses added from the start rather than at 80%. Each variant would be evaluated on the full benchmark suite from the paper—classification, retrieval, dense prediction, localization, VLM transfer, and fairness—to attribute the 2-point ImageNet gain, the 18-point RefCOCO gain, and the 5-point PASCAL gain to specific components. This would transform SigLIP 2 from a single data point into a design manual for practitioners who want to adopt parts of the recipe without reproducing it in full.

Sensitivity analysis of the 80% self-supervised staging decision. The paper adds SILC/TIPS losses at 80% of training completion without ablating this choice or providing a theoretical justification. A targeted experiment would train multiple SigLIP 2 variants with self-supervised losses introduced at 0%, 25%, 50%, 80%, and 95% of the training schedule, measuring performance on both global tasks (ImageNet-1k zero-shot, COCO retrieval) and local tasks (PASCAL segmentation, ADE20k, RefCOCO). The hypothesis—that early introduction disrupts global alignment while late introduction fails to fully shape patch features—generates a testable prediction: there should be an inverted-U relationship with a peak near 80%. Finding that the optimal introduction point varies by model scale would reveal a new axis of scaling behavior; finding that it is insensitive to timing would simplify the recipe for practitioners.

Scaling the NaFlex variant to include self-supervised losses. The paper's NaFlex variant omits self-supervised losses for computational practicality, which limits its performance on natural image benchmarks—particularly for the B/16 model where the gap to the standard variant is substantial (ImageNet-1k: 76.2% NaFlex vs. 78.2% standard at comparable resolutions, Figure 3). A follow-up would implement self-distillation and masked prediction for NaFlex by developing an efficient multi-resolution augmentation pipeline (generating local views at the same aspect ratio as the global view, handling variable patch counts in the self-supervised losses) and measure whether adding these losses closes the gap to the fixed-resolution variant. If successful, this would produce a single checkpoint that matches or exceeds dedicated checkpoints across all resolutions and task types—a genuinely practical model for deployments requiring flexible inference.

Cross-model-family replication of the unified recipe. All SigLIP 2 results are from models trained on Google's proprietary WebLI dataset using TPU infrastructure. A replication study would implement the full SigLIP 2 recipe (sigmoid loss, LocCa-style decoder, SILC/TIPS self-supervised losses at 80%, 90/10 language mixture where applicable, debiasing filters) using open data (LAION-5B or DataComp-1B) and open infrastructure, training a ViT-L/16 model at 256px, and evaluating on the same benchmark suite. This would answer the critical generalizability question: are the gains specific to WebLI's data distribution and Google's training stack, or does the recipe transfer? Given that the paper's baselines (OpenCLIP, MetaCLIP, DFN) were trained on different data, the current results cannot distinguish recipe effects from data effects. A successful replication on LAION would make the recipe actionable for the broader open-source community; a failure would reveal important data-recipe interactions that the current paper cannot identify.

Difficulty-adaptive loss weighting during training. The paper uses fixed per-model-size loss weights for the self-supervised terms (0.25, 0.5, 1.0, 0.5 for B, L, So400m, g) without justification. A more principled approach would dynamically adjust loss weights based on training progress: for example, starting with zero self-supervised weight and gradually increasing it following a schedule, or using gradient magnitude statistics to balance the contribution of each loss to the shared vision encoder parameters. A follow-up could compare the fixed-weight approach against gradient-based adaptive weighting (e.g., GradNorm, uncertainty weighting) and a cosine schedule for the self-supervised loss weight, measuring final performance and training stability. The finding that adaptive weighting either matches or improves upon the hand-tuned weights would simplify the recipe by removing a per-model-size hyperparameter; the finding that hand-tuning is necessary would indicate that loss interaction is more complex than the paper's "orthogonal axes" framing suggests.

Characterizing the pretraining-inference compute tradeoff for vision encoders. The reference paper provides a detailed FLOPs-matched analysis of pretraining vs. test-time compute for LLMs on math reasoning. A natural extension for SigLIP 2 would be a similar analysis for vision-language encoders: given a fixed total FLOPs budget, is it better to train a larger SigLIP 2 and deploy it at low resolution, or train a smaller SigLIP 2 and deploy it at high resolution with NaFlex? The experiment would train ViT-B/16, L/16, and So400m/14 SigLIP 2 models while tracking total training FLOPs, then evaluate each at multiple resolutions (256, 384, 512, 768, 1024) on the full benchmark suite, computing total inference FLOPs per image. The resulting FLOPs-matched curves would show, for each task family, which combination of model size and inference resolution is Pareto-optimal. This would provide the same kind of principled resource allocation guidance for vision encoders that the reference paper provides for LLM reasoning, directly informing deployment decisions for production VLMs.

Practical Applications and Downstream Use Cases

Global-scale multilingual visual search and content moderation. SigLIP 2's combination of strong English performance (L/16 256px: 82.5% ImageNet-1k, 54.7% COCO T→I recall@1, Table 1) with Crossmodal-3600 multilingual retrieval coverage across 36 languages (46.5% average T→I recall@1, Figure 2) makes it immediately deployable as a single-model solution for applications that previously required separate English and multilingual encoders. A photo search engine serving queries in French, Hindi, Japanese, and Arabic can use one SigLIP 2 model for all languages rather than routing to language-specific backends, reducing infrastructure complexity and avoiding the latency cost of language detection and routing. Content moderation systems that must screen images against text-based policies (e.g., detecting policy-violating content described in local language) benefit from the same unification. The practical savings come from eliminating the 2-3× model serving cost of maintaining separate English and multilingual models, and from avoiding the ~16 percentage point multilingual retrieval gap between SigLIP v1 (30.9% T→I) and SigLIP 2 (46.5%) that would make English-only models unusable for non-English queries.

Document understanding and OCR in vision-language models. SigLIP 2's strong performance as a frozen vision encoder for VLMs on text-heavy benchmarks—achieving 74.0% on TextVQA and 65.9% on DocVQA at So400m/14 384px (Table 6, Figure 4)—makes it a compelling backbone for applications involving document digitization, receipt parsing, form understanding, and screenshot-to-text conversion. The 4.3 percentage point TextVQA improvement over SigLIP v1 at the same resolution (74.0% vs. 69.7% for So400m/14 384px) and the dramatic 18.7 percentage point RefCOCO val improvement (86.04% vs. 67.33% for L/16 256px, Table 5) mean that VLMs built on SigLIP 2 can both read text in images and spatially ground that text—answering "what does the third line of the invoice say?" rather than just "what text is in this image?" For enterprise document processing pipelines, the NaFlex variant adds resolution flexibility: a single checkpoint can process standard documents at 256px sequence length for speed and high-density documents at 1024px for accuracy, without deploying separate models. The measured benefit is that NaFlex achieves 25.1% SciCap T→I at 1024 tokens versus the standard variant's 25.2%, but at low resolution (64 tokens) NaFlex reaches 5.2% versus the standard variant's (unevaluated, but likely near-zero) performance (Table 7), enabling the system to operate across a wider range of quality-vs-latency tradeoffs from a single model.

Training data generation for fine-grained visual instruction tuning. SigLIP 2's referring expression comprehension accuracy (86.04% on RefCOCO val for L/16 256px, Table 5) and grounded captioning capability (via the pretrained but unreleased LocCa decoder) make it suitable for automatically generating spatially-grounded training data for VLMs. A pipeline could use SigLIP 2 to: (1) detect objects in images using open-vocabulary detection (42.8% COCO AP for B/16, 32.7% LVIS APr, Table 4), (2) generate referring expressions describing those objects ("the red cup on the left side of the wooden table"), and (3) produce grounded captions for specific regions. The generated data could then fine-tune an instruction-following VLM (like PaliGemma or LLaVA) to improve its spatial reasoning and grounding capabilities without requiring expensive human annotation. The key numbers supporting this use case: the 42.8% COCO AP and 32.7% LVIS APr for B/16 (Table 4) provide a baseline detection capability that can be scaled up with larger models, and the 86.04% RefCOCO val accuracy (Table 5) indicates that the generated referring expressions will be correctly grounded when used as training targets.

Fairness-aware visual recognition in sensitive applications. SigLIP 2's dramatic reduction in representation bias—from 35.5% to 7.3% for L/16 256px (Figure 6, Table 9)—makes it the preferred vision encoder for applications where gender or demographic biases in model outputs are legally or ethically unacceptable. Hiring tools that analyze resume photos, educational platforms that classify student-submitted images, and healthcare applications that triage medical images all face scrutiny over biased outputs. SigLIP 2's 28.2 percentage point reduction in gender association bias (for L/16) means that images of objects are far less likely to be spuriously associated with a particular gender, reducing the risk of downstream harm. The caveat from Table 9 is that income-level performance disparity only modestly improves (32.0% → 31.1% for L/16 256px), so the fairness improvement is concentrated in representation bias rather than performance equity—practitioners should understand this scope when making deployment decisions. For applications where representation bias is the primary concern (e.g., captioning systems that should not assume "nurse" implies female or "CEO" implies male), SigLIP 2's debiasing filters provide concrete, measurable benefits over prior open-weight models.

When to Prefer This Method

Prefer SigLIP 2 over SigLIP v1 when:

• You need a single model that performs well on both English and multilingual benchmarks—SigLIP 2 L/16 256px achieves 82.5% ImageNet-1k (vs. 80.5% for SigLIP v1) while reaching 46.5% Crossmodal-3600 T→I (vs. 30.9% for SigLIP v1), eliminating the need for separate English and multilingual encoders.
• Your application involves localization tasks (referring expression comprehension, grounded captioning, open-vocabulary detection)—SigLIP 2's 18.7 percentage point RefCOCO val improvement over SigLIP v1 (Table 5) is transformative for these use cases.
• You need dense features for segmentation, depth estimation, or related tasks—SigLIP 2 So400m/14 224px achieves 77.1% PASCAL mIoU vs. 72.0% for SigLIP v1 (Table 2).
• You are deploying in fairness-sensitive contexts and need reduced representation bias—SigLIP 2 L/16 256px has 7.3% representation bias vs. 35.5% for SigLIP v1 (Figure 6).

Prefer SigLIP 2 over LocCa when:

• You need a model that works across both localization and global semantic tasks—LocCa achieves 88.34% on RefCOCO val vs. SigLIP 2's 86.04% (Table 5), but LocCa is trained on English-only data and does not provide contrastive alignment for retrieval or zero-shot classification. SigLIP 2 sacrifices 2.3 points of RefCOCO accuracy for broad multilingual and semantic capabilities.
• The 2.3 point localization gap may close if the pretrained LocCa decoder is used for initialization rather than training a new decoder from scratch—the paper explicitly notes this as expected future improvement.

Prefer the standard fixed-resolution SigLIP 2 variant over NaFlex when:

• Your deployment can afford separate checkpoints per resolution and you primarily handle natural images—the standard B/16 variant achieves 78.2% ImageNet-1k vs. NaFlex's 76.2% at comparable resolutions (Table 7) due to the active data curation and self-supervised losses that NaFlex omits.
• You are deploying a B/16 or B/32 model where the performance gap from omitted components is largest.

Prefer NaFlex over the fixed-resolution variant when:

• Your application involves document understanding, OCR, or UI screenshots where aspect ratio distortion is most damaging—NaFlex B/16 achieves 7.4% HierText T→I at sequence length 256 vs. the standard variant's 6.1% (Table 7), a 21% relative improvement.
• You need to serve multiple resolutions from a single checkpoint to minimize storage and deployment complexity, and your model size is So400m or larger where the performance gap to the standard variant is minimal (Figure 3, Table 7).
• You are deploying a large model (So400m or g) where the self-supervised and distillation omissions have negligible impact—the paper shows the two variants are nearly identical for So400m at sequence lengths 256-1024 (Table 7).

Prefer scaling pretraining over adopting SigLIP 2 when:

• Your base model completely lacks the capabilities you need (analogous to difficulty bin 5 in the reference paper)—SigLIP 2 improves existing capabilities but does not create new ones. The paper provides no evidence that tasks on which SigLIP v1 performs at chance level become solvable with the improved recipe; the gains are on tasks where SigLIP v1 already had non-trivial performance.
• You have sufficient compute to train a much larger model from scratch and your primary metrics are English-focused classification and retrieval—the paper does not provide a FLOPs-matched comparison showing that SigLIP 2's recipe is more efficient than simply scaling up a simpler recipe. Without such analysis, a practitioner who can afford to train a ViT-g with the SigLIP v1 recipe may achieve similar or better English performance than SigLIP 2 at a smaller scale, though they would sacrifice multilingual, localization, and fairness capabilities.