Dual-View Training for Instruction-Following Information Retrieval

🎯 Pitch

Retrieval models typically ignore the fine print: they match on topic, not on user constraints. This paper exploits a clever polarity-reversal trick: for every hard negative that matches the query but violates an instruction, an LLM generates a complementary instruction that makes that same document the correct answer, forcing the retriever to actually read and obey the constraints.

1. Executive Summary

This paper proposes a dual-view data synthesis strategy based on polarity reversal for instruction-following information retrieval (IF-IR), where an LLM generates complementary instructions that swap the relevance labels of existing document pairs — so that a document which is an instruction-negative under the original instruction becomes the positive under the new one, and vice versa (e.g., a passage about subglacial volcano formation that is relevant under one instruction becomes excluded under a synthesized instruction requiring discussion of environmental impacts rather than physical formation). By presenting the same query-document pair under two instructions with inverted relevance, the training signal forces a 305M-parameter bi-encoder to condition its representations on instruction semantics rather than fixed topical cues, improving FollowIR p-MRR by 45% (from 5.21 to 7.57) and surpassing general-purpose embedding models of comparable or larger scale. The paper further establishes that data diversity and instruction supervision serve complementary roles — the former sustains general retrieval quality while the latter drives instruction sensitivity — and that dual-view synthesis resolves the tension between them, but only when the synthesized pairs replace non-instruct data rather than being added on top, since mixing non-instruct samples can dilute the instruction signal and cause the retriever to contradict instruction-defined relevance entirely (yielding negative p-MRR scores on InfoSearch).

2. Context and Motivation

The Core Problem: Retrieval Models That Ignore Instructions

The fundamental challenge this paper tackles is that modern dense retrieval models, despite their sophistication at semantic matching, systematically fail to obey explicit user constraints. Instruction-following information retrieval (IF-IR) extends traditional search by requiring systems to honor not just a natural-language query but also an accompanying instruction specification that defines what kind of relevance the user wants — constraints on document attributes like writing style, required sections, geographic scope, time period, evidence type, or excluded topics. A user might ask for "volcanoes" but specify that documents must discuss environmental impacts and must not describe physical formation mechanisms. A semantically capable retriever might find many documents about volcanoes, but it must also enforce the instruction's inclusion/exclusion criteria.

This is not an edge case or a niche application. Instruction-conditioned retrieval has emerged as a dominant paradigm precisely because it mirrors how humans actually search: we rarely want everything about a topic; we want specific aspects, formats, or perspectives. The rise of instruction-tuned text embeddings (Su et al., 2023) and retrieval-augmented generation (RAG) pipelines has made instruction-following a practical necessity — every time a chat system formulates a search query to retrieve supporting documents, the instruction specifies what kind of evidence is needed, and retrieval failure cascades into downstream generation failure.

The paper cites Weller et al. (2025a) as having provided the first systematic quantification of just how badly current models fail at this task. Their FollowIR benchmark uses human-annotated instruction pairs where the same query is paired with two different instructions that fundamentally alter which documents should rank highly. The p-MRR metric they introduce measures whether a retriever correctly adjusts its rankings when the instruction changes: positive p-MRR means the model ranks the right document higher under each instruction; zero means the model ignores the instruction entirely; negative p-MRR means the model actively contradicts the instruction by ranking documents in the opposite direction from what the instruction specifies. The paper's Section 1 reports that Weller et al.'s evaluation found most current models fail to internalize detailed relevance criteria, relying instead on superficial query-document similarity and largely ignoring the specific constraints imposed by instructions. This is the empirical gap — retrievers don't just underperform on IF tasks; they often behave as if instructions are irrelevant noise.

Why This Problem Matters: RAG Pipelines and Real-World Deployment

The significance of this failure mode extends beyond benchmark numbers. In a RAG system, the retrieval step acts as a bottleneck: if the wrong documents are retrieved because the system ignored the instruction about required evidence type, format, or perspective, the subsequent generation — no matter how powerful the LLM — produces outputs grounded in wrong or inappropriately framed information. Instruction-following retrieval failures are therefore multiplied through the pipeline.

The paper's Introduction frames this in terms of deployment requirements: instruction-following IR "rigorously tests the capacity of dense retrievers to adapt their behavior based on dynamic in-context directives, going beyond static notions of relevance." The word "dynamic" is key — instructions are variable, user-specific, and cannot be baked into a static relevance model. A retrieval system that works by memorizing fixed query-document associations will inevitably fail when the instruction changes what "relevant" means for the same underlying topic.

There's also a theoretical significance to this problem: IF-IR probes whether dense retrieval models actually learn to condition their representations on linguistic context or whether they primarily exploit distributional statistics of co-occurrence. The finding that strong general-purpose models like Qwen3-Embedding-0.6B (305M parameters) and EmbeddingGemma-300M achieve FollowIR p-MRR scores of only 5.09 and 5.61 (Table 1) — barely above random — suggests that even models trained on massive corpora with sophisticated objectives may not have learned to genuinely compose query and instruction semantics. The problem is thus not just practical but reveals something about the limitations of current contrastive pretraining paradigms.

Where Existing Approaches Fall Short

The paper identifies specific, concrete limitations in prior work that go beyond "models don't follow instructions well."

Promptriever's instruction-negative approach uses data inefficiently. Weller et al. (2025b) introduced Promptriever, which demonstrated that training with instruction negatives — documents that are semantically relevant to the query but become irrelevant once the instruction is applied — substantially outperforms training with standard hard negatives. The intuition is clean: standard hard negatives teach the model to distinguish relevant from irrelevant topically, while instruction negatives teach it to distinguish instruction-satisfying from instruction-violating documents within the set of topically relevant items. However, the paper argues that this approach treats instruction negatives as fixed failures — documents that should not be retrieved under the given instruction — and does not exploit the latent information that each instruction negative implicitly defines a complementary instruction under which it would be the correct document to retrieve. Section 5 explicitly contrasts the two framings: Promptriever treats negatives as what's excluded, while polarity reversal treats them as what's conditionally relevant under a different instruction.

InF-IR achieves competitive IF with small data, but doesn't address the data mixing tension. Zhuang et al. (2025) showed that IF capability can be achieved with as few as ~38k specialized triplets, suggesting that instruction-following is more about signal purity than data volume. However, the paper's Section 5 notes that this approach doesn't address what happens when IF data is mixed with standard retrieval data — the data mixing tension that the current paper identifies as a central challenge. InF-IR shows you can build an IF-specialized model with pure instruction data; the current paper shows what happens when you need a model that is both generally capable and instruction-aware, and that naively mixing the two data sources catastrophically degrades IF performance.

General-purpose embedding models fail on IF despite massive scale. The paper's Table 1 directly shows that EmbeddingGemma-300M and Qwen3-Embedding-0.6B, despite being strong general-purpose models, achieve FollowIR p-MRR scores that are comparable to or below Ins-orig (5.61 and 5.09 vs. 5.21). Critically, the paper's Section 5 describes this not as a capacity problem but as a gradient conflict: non-instruct training samples "provide gradient signal that rewards query-correlated retrieval regardless of instructions," and at typical mixing ratios this signal "overwhelms the instruction signal." Instruction sensitivity is thus framed as "a fragile property requiring consistent supervision" rather than something that emerges naturally from scale. This explains the empirical puzzle of why larger models don't necessarily follow instructions better — they've received more gradient signal toward instruction-agnostic retrieval.

No existing approach simultaneously maintains general retrieval quality and instruction sensitivity. This is the key integrative gap. Promptriever showed instruction data helps IF. InF-IR showed pure instruction data works well. But neither demonstrated a scalable recipe that preserves strong general retrieval performance while achieving competitive IF, because pushing hard on either axis seems to degrade the other. The paper's central empirical finding — the data mixing tension — hadn't been systematically characterized before this work. The finding that supplementing instruction data with non-instruct data (All-orig in Table 1) causes p-MRR to plummet into negative territory (−23.22 and −49.65 on InfoSearch) is novel and has direct implications for anyone building retrieval models that must serve both general and instruction-conditioned queries.

How This Paper Positions Itself

The paper positions itself not as proposing an entirely new training paradigm but as offering a data-centric solution to a data mixing problem. The contribution is the observation that instruction negatives are not waste products but untapped resources — each one defines a complementary instruction, and synthesizing that instruction creates a training pair that simultaneously provides instruction supervision (because the model must learn to switch its ranking based on instruction content) and data diversity (because the same document pair appears in two contexts with opposite relevance labels). Section 5 frames this succinctly: "Polarity reversal reframes this: an instruction negative is a conditionally relevant document, one that should be retrieved under a different, complementary instruction."

The paper explicitly contrasts this with Promptriever's approach: "Both Promptriever [...] and InF-IR [...] demonstrate that instruction-tied negatives outperform generic hard negatives, but treat negatives as fixed failures relative to a given instruction, i.e., documents that should not be retrieved under it." The innovation is moving from instructional exclusions (what you should NOT retrieve under this instruction) to instructional distinctions (what makes these two instructions different, such that they demand opposite documents). This is described as "a structurally richer supervisory signal" because it directly targets the representational property that IF requires — the ability to encode how two instructions diverge, not just which documents each excludes.

The paper further positions its approach as resolving the tension between the two training data strategies that the experiments systematically characterize: instruction-only data (good IF, poor general quality because of limited diversity) versus mixed data (good general quality, catastrophic IF degradation because of gradient conflict). Dual-view synthesis resolves this tension "by providing instruction-conditioned training pairs at scale, simultaneously maintaining the data volume that sustains general quality and the instruction signal that drives IF capability." The data volume point is important because the paper finds that "data volume, rather than source heterogeneity per se, is the primary driver of general retrieval quality" — so the DV pairs provide volume without introducing non-instruction-conditioned samples that would corrupt the gradient signal.

A final positioning point: the method is designed to be costless and pipeline-compatible. Section 6 emphasizes that "the approach requires no changes to existing pipelines" — the synthesized instructions are generated once via LLM prompting and then used as standard training data in a contrastive InfoNCE loss. There is no architectural modification, no loss function change, no curriculum learning, no additional inference overhead. This is a deliberate positioning choice: rather than asking practitioners to adopt a new training framework, the paper offers a data augmentation technique that can be dropped into existing contrastive training pipelines as a simple replacement for or supplement to current data mixtures. The claim is that the right data, not the right architecture or objective, is the bottleneck, and the evidence from the experiments supports this — the same model architecture, same optimizer, same temperature, same number of hard negatives, achieves dramatically different IF performance purely as a function of data composition.

3. Technical Approach

3.1 Reader Orientation

This is a data-centric paper whose core idea is that instruction-following retrieval failures can be addressed not by changing the model architecture or training objective, but by synthesizing training data that explicitly forces the model to condition its representations on instruction content. The system being built is a standard bi-encoder dense retrieval model trained with contrastive InfoNCE loss — what changes is the composition of the training data: rather than presenting each document as fixed-relevance given a query-instruction pair, the method creates paired training instances where the same two documents appear with opposite relevance labels under complementary instructions, forcing the model to learn that relevance is a function of the instruction, not of the document text alone.

The problem it solves is the instruction blindness documented by Weller et al. (2025a): current dense retrievers largely ignore instructions and rely on superficial query-document topical similarity. The solution's shape is a data augmentation pipeline that takes existing training triplets (query, instruction, positive/negative documents) and uses an LLM to generate a new instruction that reverses which document is relevant, creating a complementary training instance that, when trained on alongside the original, imposes a direct optimization pressure against instruction-agnostic representations.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components, arranged as a data synthesis pipeline feeding a standard contrastive training loop:

Seed Data Selector: Takes the Promptriever training dataset and identifies instances that contain at least one instruction negative — a document that is topically relevant to the query (it discusses the right subject) but violates the instruction's constraints (wrong format, wrong scope, wrong perspective, etc.). These instances become the input to the synthesis step. The selector's job is to filter for data points that have the right structure (query, instruction, positive document, at least one instruction-negative document) to make polarity reversal meaningful.
LLM-Based Instruction Synthesizer: Given a query, an original instruction, a positive document, and a specific instruction-negative document, an LLM (Qwen3-Next-80B-A3B-Instruct) is prompted to generate a new instruction that is coherent with the query but reverses the relevance polarity — so the original positive becomes an instruction-negative under the new instruction, and the original instruction-negative becomes the new positive. The LLM receives detailed guidance about what makes a valid reversal (attribute-based constraints, diversity from the original, conciseness) but does not modify the query or document texts. Output is a single new instruction string.
Dual-View Batch Constructor: During training, each augmented instance contributes two training data points to the batch — the original view (query + original instruction → original positive, original negatives including the instruction-negative) and the polarity-reversed view (query + new instruction → original instruction-negative as positive, original positive as instruction-negative, remaining negatives unchanged). This means the model sees the same document pair under two instructions with opposite relevance labels within the same training batch (since batches are constructed randomly from the full training set, not explicitly paired — the dual views coexist as independent samples).
Standard Contrastive Training Loop: A bi-encoder (gte-multilingual-mlm-base, 305M parameters) encodes query+instruction concatenations through one tower and document texts through another, computes cosine similarity for all query-document pairs in a batch, and optimizes InfoNCE loss with temperature τ = 0.02 using 30 hard negatives per query (including 1–3 instruction negatives). The training loop is completely unchanged from standard practice — the innovation is entirely in the data that feeds it.

Information flows sequentially: Promptriever data → selector filters for instruction-negative-containing instances → LLM synthesizer generates complementary instructions → dual-view constructor creates paired training instances → contrastive training loop optimizes the bi-encoder. At inference time, there are no changes: the trained encoder takes query+instruction concatenated, produces an embedding, and retrieves nearest-neighbor documents — the dual-view training signal has shaped the representation space but doesn't require any runtime modification.

3.3 Roadmap for the Deep Dive

First, the formal problem setup and contrastive training objective, including how queries and instructions are combined, how relevance is defined, and the InfoNCE loss — since all subsequent data manipulations serve to modify what this objective optimizes for.
Second, the core polarity reversal mechanism: what it means for relevance to reverse, why instruction negatives are the key enabling resource, and the formal properties the synthesized instruction must satisfy for the dual-view constraint to be valid.
Third, the LLM prompting pipeline for instruction synthesis in full detail — the prompt template structure, the reasoning steps the LLM is instructed to follow, the guardrails that prevent degenerate outputs, and the quality control outcomes (99%+ usable rate).
Fourth, the dual-view training dynamics: what happens when both views appear in training, why this penalizes instruction-agnostic representations, and the gradient-level explanation for why this signal is stronger than simple instruction-negative training.
Fifth, the experimental data configurations (Ins-orig, Ins-DV, All-orig, All-DV) — what each configuration contains, how they are size-matched, and what each comparison is designed to isolate (the effect of dual-view data, the effect of data mixing, and the interaction between them).

3.4 Detailed, Sentence-Based Technical Breakdown

This is a data synthesis and training methodology paper whose core idea is that generating complementary instructions that swap document relevance labels creates a training signal that forces bi-encoder retrievers to genuinely condition on instruction semantics rather than relying on fixed topical associations. The method requires no architectural changes, no loss function modifications, and no inference-time overhead — it operates entirely by changing the composition of the contrastive training data.

Problem Formalization and Contrastive Training Objective

The paper operates within the standard bi-encoder dense retrieval framework. Formally, given a query $q$ and an instruction $I$ , the system concatenates them into a joint query representation $q \oplus I$ and encodes it through a transformer encoder $E_Q$ to produce a query embedding $\mathbf{e}_q = E_Q(q \oplus I)$ . Each candidate document $d$ is independently encoded through a document encoder $E_D$ (which may share weights with $E_Q$ or be separate) to produce a document embedding $\mathbf{e}_d = E_D(d)$ . The relevance score between the query-instruction pair and a document is the cosine similarity:

$s(q \oplus I, d) = \frac{\mathbf{e}_q \cdot \mathbf{e}_d}{\|\mathbf{e}_q\| \|\mathbf{e}_d\|}$

where $\mathbf{e}_q$ is the query-instruction embedding, $\mathbf{e}_d$ is the document embedding, $\|\cdot\|$ denotes L2 norm, and the dot product is taken over the embedding dimension (typically 768 for the base encoder used in experiments).

What it computes: a scalar between −1 and 1 measuring the angular alignment between the query-instruction representation and the document representation in the shared embedding space. Higher values indicate the retriever judges the document more relevant to the query under the given instruction.

Why this form: cosine similarity is the standard scoring function for bi-encoder retrieval because it normalizes for embedding magnitude, preventing models from artificially increasing scores by producing longer-norm embeddings for certain inputs. In production retrieval, cosine similarity enables efficient approximate nearest neighbor search via maximum inner product search (MIPS) on L2-normalized vectors, since $\|\mathbf{e}_q\| = \|\mathbf{e}_d\| = 1$ after normalization and cosine similarity equals dot product.

During training, the model receives batches of queries and documents with known relevance labels. For each query $q \oplus I$ , there is one positive document $d^+$ (annotated as relevant under that instruction) and a set of negative documents $d^-_1, \ldots, d^-_K$ (annotated as not relevant, where $K = 30$ in this paper's experiments). The negatives include both standard hard negatives (topically similar but not satisfying the instruction) and instruction negatives (topically relevant but violating the instruction's specific constraints). The model is trained with the InfoNCE loss:

$\mathcal{L}_{\text{InfoNCE}} = -\log \frac{\exp(s(q \oplus I, d^+) / \tau)}{\exp(s(q \oplus I, d^+) / \tau) + \sum_{j=1}^{K} \exp(s(q \oplus I, d^-_j) / \tau)}$

where $s(q \oplus I, d)$ is the cosine similarity defined above, $\tau = 0.02$ is the temperature parameter, $d^+$ is the positive document, and $d^-_1, \ldots, d^-_K$ are the $K = 30$ negative documents for this query.

What it computes: the negative log-likelihood that the model assigns the highest similarity score to the positive document among all $K+1$ candidates. The temperature $\tau$ controls the sharpness of the softmax distribution — lower temperatures make the loss more sensitive to small similarity differences, effectively increasing the penalty for ranking negatives above positives. The sum in the denominator runs over all negatives in the batch for that query.

Why this form: InfoNCE is the standard contrastive objective for representation learning because it directly optimizes for the property needed at inference time — the ability to rank the relevant document above all irrelevant ones. The $-\log$ form makes the objective a proper scoring rule that is minimized when the positive document's softmax probability approaches 1. The temperature $\tau = 0.02$ is a relatively low value (standard range is 0.01–0.1), which means the model is heavily penalized for even small overlaps in similarity between positives and negatives — this sharpens the representation space and is standard practice in embedding model training. The choice of 30 negatives per query balances computational efficiency (processing all pairs in a batch scales quadratically) with sufficient contrastive signal — too few negatives and the model doesn't learn fine-grained distinctions; too many and the batch size becomes prohibitive.

Critical design detail: query-instruction concatenation. The query and instruction are concatenated before encoding (Section 3, Training Details): "For both encoders, the query and instruction are concatenated before encoding, while documents are encoded independently." This means the query encoder receives a single string like "Query: [q] Instruction: [I]" and produces one embedding. The practical implication is that the instruction semantics and query semantics are fused at the token level inside the transformer — there is no separate encoding of query and instruction followed by fusion. This is the standard approach in instruction-tuned embedding models (following Su et al., 2023 and Weller et al., 2025b) and means that the model must learn to compose query and instruction information within a single forward pass, rather than computing separate embeddings and combining them post-hoc. The max sequence length is 512 tokens for both query+instruction and documents.

The Core Polarity Reversal Mechanism

The central insight of the paper is that the Promptriever dataset (Weller et al., 2025b) contains a resource that had been used only partially: instruction negatives. An instruction negative is defined formally as a document $d^-_{\text{inst}}$ that satisfies two properties simultaneously: (1) it is relevant to the pure query $q$ without any instruction applied — it discusses the right topic, uses relevant terminology, and would be considered a reasonable search result for the query alone; but (2) it violates one or more constraints specified in the instruction $I_{\text{orig}}$ — it might discuss the right phenomenon but from the wrong geographic region, or use the wrong evidence type, or lack a required section, or fall outside a specified time window. The existence of such documents in the Promptriever training data means that for many queries, there is a known document $d^-_{\text{inst}}$ that is topically on-target but instructionally wrong.

The key observation is that if a document $d^-_{\text{inst}}$ is relevant to the pure query but excluded by $I_{\text{orig}}$ , then there implicitly exists some instruction $I_{\text{new}}$ under which $d^-_{\text{inst}}$ would be relevant and $d^+_{\text{orig}}$ (the original positive) would be excluded. This is not a guaranteed property for every instruction negative — if the negative is irrelevant to the query in any sense, no instruction can make it relevant — but when the instruction negative is truly a "wrong aspect" rather than a "wrong topic," the reversal is possible by defining an instruction that favors the negative's attributes and excludes the positive's attributes.

The paper formalizes this as a polarity reversal condition. Given a query $q$ , an original instruction $I_{\text{orig}}$ , a positive document $d^+$ (relevant under $I_{\text{orig}}$ ), and an instruction negative $d^-_{\text{inst}}$ (topically relevant to $q$ but not relevant under $I_{\text{orig}}$ ), the synthesized instruction $I_{\text{new}}$ must satisfy:

Coherence: $I_{\text{new}}$ must be semantically coherent with $q$ — it must be a sensible instruction to pair with this query, not a nonsensical or contradictory directive.
Complementarity: $I_{\text{new}}$ must be "sufficiently distinct" from $I_{\text{orig}}$ (Section 2) — it must not be a trivial rewording of the original instruction that would merely restate the same constraints. The goal is genuinely different relevance criteria, not paraphrasing.
Relevance under new instruction: $d^-_{\text{inst}}$ must be relevant under $I_{\text{new}}$ — it must satisfy the new instruction's constraints and be a reasonable document to retrieve.
Exclusion under new instruction: $d^+$ must become an instruction negative under $I_{\text{new}}$ — it must violate one or more of the new instruction's constraints, such that it would be filtered out.
Preservation of remaining negatives: Any other instruction negatives $N_1, \ldots, N_k$ associated with the original instance must remain instruction negatives under $I_{\text{new}}$ — they must not accidentally become positive under the new instruction, which would create a training inconsistency.

The final condition is particularly important for training data quality. If the synthesized instruction accidentally made one of the other instruction negatives positive, the training instance would contain competing positive signals, and the contrastive loss would receive conflicting gradients. The LLM is explicitly instructed to verify that all other negatives remain excluded (the "Sanity checks" step in the prompt template, Appendix A).

Figure 1 in the paper provides a concrete illustration of this reversal with a volcano-related query. The original instruction asks for documents that "describes a specific type of volcano that has not been directly observed erupting, and provides information about its formation or characteristics." The positive document under this instruction discusses subglacial volcanoes, their formation mechanisms, and why they haven't been directly observed. The instruction negative discusses the climatic impacts of volcanic eruptions (Mount Tambora, etc.) — it mentions unwitnessed eruption types but does not describe physical formation, so it's excluded by the original instruction's requirement for formation/characteristics information. The synthesized new instruction flips this: it asks for documents that "discuss the broader environmental impacts or research insights associated with unwitnessed eruption types, strictly excluding descriptions of the volcano's physical formation or structural classification." Now the climate impacts document becomes relevant (it discusses environmental impacts) and the subglacial volcano document becomes excluded (it focuses on physical formation, which the new instruction explicitly prohibits).

Why this mechanism targets the root cause of instruction blindness. The fundamental reason retrievers fail at IF-IR is that they learn to associate queries with documents based on topical co-occurrence statistics — "volcanoes" co-occurs with "subglacial volcanoes" and "volcanic eruptions" and "Mount Tambora" in training, so the model learns to give all of them high similarity to the query regardless of the instruction. Standard instruction-negative training partially addresses this by teaching the model that some documents should be ranked low under a given instruction, but this is a unidirectional signal: it teaches exclusion but not the conditional nature of relevance. The model learns " $d^-_{\text{inst}}$ is wrong under $I_{\text{orig}}$ " but does not learn that " $d^-_{\text{inst}}$ is right under some other instruction, and $d^+$ would be wrong under that instruction."

Polarity reversal converts this into a bidirectional signal. When the model sees both $(q \oplus I_{\text{orig}}, d^+ \text{ positive}, d^-_{\text{inst}} \text{ negative})$ and $(q \oplus I_{\text{new}}, d^-_{\text{inst}} \text{ positive}, d^+ \text{ negative})$ in training, the only way to satisfy both constraints simultaneously is to encode $q \oplus I_{\text{orig}}$ and $q \oplus I_{\text{new}}$ as substantially different vectors, such that one is closer to $d^+$ and the other is closer to $d^-_{\text{inst}}$ . If the model were instruction-agnostic — if it encoded all instructions as essentially the same vector for a given query — then $q \oplus I_{\text{orig}}$ and $q \oplus I_{\text{new}}$ would be nearly identical, and the model would receive contradictory gradients: it would be pushed to rank $d^+$ above $d^-_{\text{inst}}$ (from the first view) and simultaneously rank $d^-_{\text{inst}}$ above $d^+$ (from the second view). The only stable solution under gradient descent is to genuinely condition the query-instruction encoding on the instruction content, separating the two instruction representations in embedding space so that each can point to its correct document.

This is what the paper means by "This dual objective directly penalizes instruction-agnostic representations: no single query encoding can satisfy both constraints unless it genuinely conditions on the semantic content of the instruction" (Section 2). The key phrase is "no single query encoding" — if the model tried to use the same embedding for both views, the InfoNCE loss would be high for at least one of them (and likely both, since the softmax would distribute probability mass across the two documents rather than concentrating it on the correct one). The gradient from the dual views therefore pushes the representations of $q \oplus I_{\text{orig}}$ and $q \oplus I_{\text{new}}$ apart in directions that align with $d^+$ and $d^-_{\text{inst}}$ respectively.

LLM-Based Instruction Synthesis Pipeline

The synthesis pipeline uses Qwen3-Next-80B-A3B-Instruct (Qwen Team, 2025) as the generator model, accessed via prompting. The choice of model is important: it needs to be capable enough to understand subtle attribute distinctions between documents and generate constraints that precisely target those distinctions, but the synthesis is done offline and the model is not fine-tuned or adapted for this task — it is used zero-shot with a detailed prompt.

Seed data selection. Not every training instance in Promptriever is suitable for polarity reversal. The paper selects only instances that contain at least one pre-existing instruction negative — a document that is topically relevant to the query but marked as negative under the original instruction. This is stated in Section 2: "We construct our seed dataset by selecting instances from the promptriever data that contain at least one pre-existing instruction negative." This filtering step is necessary because polarity reversal requires a document whose relevance status can be flipped; a standard hard negative (topically irrelevant to begin with) cannot become relevant under any reasonable instruction, so there is no meaningful complementary instruction to synthesize.

For each qualifying instance, the paper generates exactly one complementary instruction. The generation uses the specific instruction negative that will become the positive under the new instruction — the prompt explicitly identifies which document is the target for reversal (called $N^*$ in the prompt template). The remaining instruction negatives $N_1, \ldots, N_k$ (if any exist beyond the targeted one) are also provided to the LLM so it can ensure they remain excluded under the new instruction.

The prompt template structure. Appendix A provides the complete prompt, which is structured as a detailed task specification with six components:

Goal statement: A clear declaration that the LLM must "create a new synthetic instruction that reverses the original relevance judgment only for the specified passages," with explicit bullet points specifying that the original positive must become an instruction negative, the specific instruction negative must become positive, and all other negatives must remain excluded.
Input fields: The query, original instruction, positive passage, specific instruction negative (the target for becoming positive), and remaining instruction negatives (array, may be empty) are all provided as separate labeled fields.
Definitions: The prompt clarifies what "relevant to the pure query" means (satisfies user intent without any instruction) and what an "instruction negative" is (relevant to the pure query but excluded by instruction constraints like scope, geography, timeframe, format, source constraints, or audience level).
Methodology steps: A five-step reasoning process the LLM is instructed to follow:
- Step 1 (Profile passages): Identify attributes of the positive passage (domain, geography, timeframe, audience, medium/format, methodology, sources, constraints) and attributes of the target instruction negative that distinguish it from the positive. Also skim remaining negatives to note attributes that must stay excluded.
- Step 2 (Choose reversal levers): Craft constraints that positively select the target negative's attributes while excluding the original positive via "≥1 hard, objective constraint." The prompt provides a list of useful constraint levers: domain narrowing, region, timeframe/recency, audience level, style/format, methodology/evidence type, required artifacts, explicit exclusions.
- Step 3 (Diversity requirement): The new instruction must differ in format and perspective from the original, avoiding trivial rewording and switching things like voice, deliverable type, or constraint style.
- Step 4 (Sanity checks): Verify that the original positive would be filtered by the new constraints, the target negative is clearly included, and all other negatives remain excluded. If any check fails, the LLM is instructed to tighten constraints or pivot toward the target negative's attributes.
- Step 5 (Conciseness): The instruction must be "one or two sentences, imperative, concrete, and unambiguous."
Guardrails: Explicit prohibitions against referencing passage IDs or the meta-task, modifying the query or passages, and requiring exclusion/inclusion by attribute only.
Output format: The LLM must reason step by step, then output the new instruction in XML tags: <answer><new_instruction>[instruction text]</new_instruction></answer>. If the task is impossible, it should output <answer>None</answer>.

The prompt design is notable for its specificity and structure. Rather than simply asking the LLM to "write a different instruction," it provides a detailed reasoning framework that mirrors how a human annotator would approach the task — identify distinguishing attributes, select constraints that target those attributes, verify exclusions hold, ensure diversity. This structured approach likely explains the high quality rate (99%+ usable per manual inspection of 100 samples) without additional filtering.

Quality control and filtering. The paper reports that "one annotator manually checked 100 datapoints and confirmed that over 99% of the DV instructions are usable. Thus, no additional filtering was conducted." This is an important detail: the synthesis pipeline produces high-quality outputs with essentially no human intervention beyond the initial prompt design. The "usable" criterion presumably means that the polarity reversal condition holds — the original positive is genuinely excluded under the new instruction, the target negative is genuinely included, and remaining negatives stay excluded — but the paper does not specify the exact grading rubric.

The fact that no additional filtering was applied means the training data includes the rare cases (less than 1%) where the synthesized instruction fails to properly reverse polarity. The paper does not analyze whether these failures harm training (they would introduce inconsistent signal, since a document that should be an instruction-negative under the new instruction might actually be positive, creating gradient conflict), but the low failure rate suggests any negative impact is negligible relative to the benefit of the 99%+ of correct dual views.

Cost considerations. The paper emphasizes that the approach requires "no additional annotation cost" (Section 6) — the synthesis uses an LLM rather than human annotators, which is both cheaper and faster. The cost is the inference compute for Qwen3-Next-80B-A3B-Instruct, which is substantial per sample (an 80B-parameter model with 3B active parameters via mixture-of-experts) but is a one-time offline cost. The paper generates approximately 480k dual-view samples for the Ins-DV configuration (240k original + 240k synthetic) and 440k for the All-DV configuration, though the exact number generated may be slightly lower since not all original training instances have instruction negatives.

Dual-View Training Dynamics

Once the complementary instructions are synthesized, the training data is constructed so that both the original and polarity-reversed views appear in the training distribution. A critical implementation detail from Section 2: "In our controlled experiments, DV samples substitute for an equal-sized portion of the training set rather than being added on top, enabling size-matched comparisons across all configurations." This means that Ins-DV does not contain 480k original + 240k synthetic = 720k samples; it contains 240k original + 240k synthetic = 480k total, matching the size of Ins-orig (480k original). Similarly, All-DV contains 440k original + 440k synthetic = 880k total, matching All-orig (440k original + 440k non-instruct). The substitution, rather than augmentation, is what enables the paper to isolate the effect of data composition from the effect of data volume.

How dual views enter training. During contrastive training, batches are constructed by random sampling from the full training set. Each dual-view instance contributes two independent data points that may appear in the same batch or different batches. The paper notes that "the model sees the same document pair under two instructions with opposite relevance labels" (paraphrased from earlier analysis), but does not specify whether these are explicitly paired in the same batch or appear randomly. The standard Arctic-Embed framework likely constructs batches by sampling queries uniformly from the training set and then sampling negatives for each query, meaning the two views of the same document pair could appear together or separately depending on random sampling. Either way, the gradient signal accumulates across training steps: the model must learn representations that satisfy both the original and polarity-reversed constraints simultaneously.

Why this penalizes instruction-agnostic representations. The gradient-level argument is central to the paper's claims and deserves careful unpacking. Consider what happens in embedding space for a simplified case with one document pair $(d^+, d^-_{\text{inst}})$ and two instructions $(I_{\text{orig}}, I_{\text{new}})$ . The InfoNCE loss for the original view pushes the query-instruction embedding $\mathbf{e}_{q \oplus I_{\text{orig}}}$ toward $\mathbf{e}_{d^+}$ and away from $\mathbf{e}_{d^-_{\text{inst}}}$ . The InfoNCE loss for the reversed view pushes $\mathbf{e}_{q \oplus I_{\text{new}}}$ toward $\mathbf{e}_{d^-_{\text{inst}}}$ and away from $\mathbf{e}_{d^+}$ . These are not contradictory because they involve different query-instruction embeddings — $\mathbf{e}_{q \oplus I_{\text{orig}}}$ and $\mathbf{e}_{q \oplus I_{\text{new}}}$ are produced by different inputs and can be different vectors.

If the encoder learns to produce $\mathbf{e}_{q \oplus I_{\text{orig}}} \approx \mathbf{e}_{d^+}$ and $\mathbf{e}_{q \oplus I_{\text{new}}} \approx \mathbf{e}_{d^-_{\text{inst}}}$ , both losses are minimized. But if the encoder is instruction-agnostic — if it produces $\mathbf{e}_{q \oplus I_{\text{orig}}} \approx \mathbf{e}_{q \oplus I_{\text{new}}}$ regardless of the instruction — then the two losses are in direct conflict. The encoder cannot simultaneously place a single embedding close to both $\mathbf{e}_{d^+}$ and $\mathbf{e}_{d^-_{\text{inst}}}$ while also keeping them far apart (as required for hard negatives in each view). The gradient from the combined loss will push the encoder to separate $\mathbf{e}_{q \oplus I_{\text{orig}}}$ and $\mathbf{e}_{q \oplus I_{\text{new}}}$ in the embedding space, which can only happen if the encoder attends to the differences between $I_{\text{orig}}$ and $I_{\text{new}}$ .

This is a more powerful signal than standard instruction-negative training because it directly measures instruction sensitivity. In standard training, the model is told what not to retrieve under $I_{\text{orig}}$ but is never told what $I_{\text{orig}}$ 's complement would retrieve. The model can satisfy the standard loss by simply learning that $d^-_{\text{inst}}$ is a "bad" document for queries about this general topic — it doesn't need to understand why it's bad or under what conditions it would be good. In dual-view training, the model must learn that $d^-_{\text{inst}}$ is conditionally relevant — good under $I_{\text{new}}$ but bad under $I_{\text{orig}}$ — which forces it to represent the conditional structure, not just the unconditional associations.

The relationship to contrastive multi-view learning. This mechanism is conceptually related to multi-view contrastive learning (e.g., SimCLR, where different augmentations of the same image are treated as positives), but with a crucial difference: in standard multi-view learning, different views of the same instance are treated as positives (they should map to similar representations), whereas in this dual-view setup, different views of the same document pair are treated as having opposite relevance labels (the positive becomes negative and vice versa). This creates a discriminative rather than invariant pressure — the model must learn to separate representations based on instruction, not make them invariant. It is closer to contrastive learning with hard negative mining (where hard negatives are those the model currently confuses with positives) but with the twist that the hard negatives are defined by instruction content rather than topical similarity alone.

Experimental Data Configurations

The paper defines four specific training data configurations that serve as the basis for all experiments. These are carefully designed to isolate the effects of dual-view data, data mixing (instruct vs. non-instruct), and their interaction. All configurations are described in Section 3 under "Backbone Models."

Ins-orig (Instruct-only, original data): This configuration uses 480k instruction-following training samples drawn entirely from the original Promptriever dataset. These are instruction-conditioned triples (query + instruction → positive document + negatives including instruction negatives). This serves as the baseline for instruction-only training — it represents the state of the art in IF-IR training data from Weller et al. (2025b). The 480k figure is the total training set size.

Ins-DV (Instruct-only, dual-view data, ours): This configuration uses 240k original instruct samples from Promptriever plus 240k dual-view synthetic samples generated by polarity reversal, for a total of 480k samples — size-matched to Ins-orig. The substitution means that half of the original instruct samples are replaced with their dual-view counterparts. The key comparison is Ins-DV vs. Ins-orig: any performance difference is attributable to the dual-view signal replacing original signal at the same data volume. The paper's hypothesis, confirmed by results, is that Ins-DV should improve IF metrics (because dual views provide stronger instruction supervision) but might degrade general retrieval (because replacing original samples reduces the diversity of training contexts — each dual-view pair uses the same query and documents as the original it replaces, just with a different instruction).

All-orig (Mixed instruct + non-instruct, original data): This configuration uses 440k instruct samples from Promptriever plus their 440k non-instruct counterparts from the original Promptriever dataset, for a total of 880k samples. The paper explains that "each instruct sample was synthesized from a corresponding non-instruct sample" (Section 3) — Promptriever's data construction process takes standard retrieval training pairs (query, positive document, hard negatives without instructions) and prompts an LLM to generate an instruction that the positive document satisfies, creating an instruct version of each non-instruct pair. So All-orig contains the paired data: both the original non-instruct version and the instruction-augmented version of each training pair. This represents the natural data mixing strategy — supplement instruction data with the non-instruct data it was derived from to increase volume and diversity. The key comparison is All-orig vs. Ins-orig: this measures the effect of adding non-instruct data on both IF and general retrieval performance.

All-DV (Mixed instruct + dual-view, no non-instruct, ours): This configuration uses 440k original instruct samples from Promptriever plus 440k dual-view synthetic samples, for a total of 880k samples — size-matched to All-orig. Crucially, this configuration contains no non-instruct data at all — all 880k samples are instruction-conditioned, with the 440k dual-view samples providing additional volume without introducing non-instruct samples. The key comparison is All-DV vs. All-orig: both have the same total volume (880k) and the same number of original instruct samples (440k), but All-DV uses dual-view pairs where All-orig uses non-instruct pairs as the supplementary data. Any performance difference isolates the effect of what the supplementary data consists of — instruction-conditioned dual views vs. instruction-free retrieval pairs.

What each comparison isolates:

Ins-DV vs. Ins-orig: The pure effect of replacing original instruct data with dual-view data at fixed volume (480k). Tests whether dual-view signal is more effective per sample than original instruct signal for IF, and at what cost to general retrieval.
All-orig vs. Ins-orig: The effect of scaling training data with non-instruct pairs. Tests whether added volume+diversity from non-instruct data helps general retrieval (hypothesis: yes) and what it does to IF (hypothesis from results: catastrophically degrades it).
All-DV vs. All-orig: The effect of using instruction-conditioned data (DV) vs. non-instruct data as the supplementary volume. Tests whether DV can provide the volume benefits of scaling (sustaining general retrieval) without the IF degradation that non-instruct mixing causes.
All-DV vs. Ins-DV: The effect of scaling instruction-conditioned data volume (880k vs. 480k). Tests whether more instruction data helps both IF and general retrieval, or whether IF saturates.

The 480k/440k discrepancy. Note that Ins- configurations use 480k instruct samples while All- configurations use 440k instruct samples. The paper does not explain this discrepancy explicitly, but the likely reason is that the All- configurations correspond to the full Promptriever dataset where every instruct sample has a paired non-instruct counterpart, and that totals 440k instruct + 440k non-instruct = 880k. The Ins- configurations use a slightly larger instruct-only subset (480k) — possibly all available instruct samples including those without paired non-instruct counterparts, or a different curation of the training data. This discrepancy means that Ins-orig and All-orig differ not only in the presence of non-instruct data but also in the number of instruct samples (480k vs. 440k), which is a minor confound. However, the paper's key comparisons are within the same size class (Ins vs. Ins, All vs. All), where the instruct count is matched, so the confound does not affect the main findings.

Why these configurations enable the paper's key claims. The paper's central argument has two parts: (1) dual-view data provides stronger instruction supervision per sample than original instruct data (demonstrated by Ins-DV > Ins-orig on IF metrics), and (2) instruction supervision and data diversity are in tension when diversity is provided by non-instruct data (demonstrated by All-orig's catastrophic IF degradation relative to Ins-orig, and All-DV's recovery of IF while maintaining general retrieval). The four configurations form a 2×2 design (Ins vs. All, orig vs. DV) that cleanly separates these effects. The fact that All-DV achieves the best of both — highest FollowIR p-MRR (8.30) and highest or second-highest Score (21.38) — while containing no non-instruct data is the paper's headline finding: dual-view synthesis resolves the data mixing tension by providing volume and instruction signal simultaneously.

4. Key Insights and Innovations

Innovation 1: Reframing Instruction Negatives as Conditionally Relevant Resources Rather Than Fixed Failures

The dominant paradigm in instruction-following retrieval training, established by Promptriever (Weller et al., 2025b), treats instruction negatives as documents that should NOT be retrieved — they are failures a model must learn to exclude. This paper's most fundamental conceptual move is to reconceptualize instruction negatives as latent positives for some complementary instruction that doesn't yet exist in the training data. This is not a minor reframing — it's a shift from what the paper calls "instructional exclusions" (learning what each instruction rules out) to "instructional distinctions" (learning what makes two instructions different such that they demand opposite documents).

What makes this intellectually distinctive is the recognition that the Promptriever dataset already contains implicit pairing information that had been treated as waste. An instruction negative isn't just a training signal for one instruction — it's a document whose attributes differ from the positive in systematic ways, and those attribute differences implicitly define a boundary between two relevance criteria. The paper's synthesis pipeline simply makes this implicit boundary explicit by generating the instruction that would place it on the other side. This is a form of data efficiency through reinterpretation: rather than collecting new labeled data or generating entirely novel training instances from scratch, the method extracts a second training instance from each existing one by prompting an LLM to articulate what the existing data already implies.

The contrast with prior work is sharp. Promptriever demonstrated that instruction negatives are valuable training signals, but used them only once — as negatives. InF-IR (Zhuang et al., 2025) showed that small amounts of pure instruction data can be highly effective, but didn't address how to scale without diluting the signal. Both approaches treat instruction negatives as consumable resources — use them once, in one direction, for one training instance. Polarity reversal treats them as reusable resources — each instruction negative can generate a new training instance by defining a complementary instruction, effectively doubling the number of instruction-conditioned training pairs available from the same underlying document set, but with the crucial property that the new pairs are not independent samples — they are structurally paired with the originals in a way that creates the dual-view training dynamic described in Section 3.

This reframing has theoretical implications beyond the immediate method. It suggests that the key bottleneck in IF-IR training is not the volume of instruction data but the density of instructional contrast — how many opportunities the model gets to see the same documents under different instructions with opposite relevance labels, forcing it to learn what distinguishes one instruction from another. Standard training with instruction negatives provides one-sided contrast (what's excluded under this instruction); dual-view training provides two-sided contrast (what's excluded under this instruction and what's included under a related but different instruction). This is a fundamentally richer learning signal, and the paper's results — 45% improvement in FollowIR p-MRR with Ins-DV over Ins-orig at the same data volume — suggest that instructional contrast density, not raw instruction data volume, is what drives IF capability.

Innovation 2: The Data Mixing Tension as a First-Class Diagnostic Concept

The paper's second major contribution is not a method but a diagnostic finding that explains a previously puzzling pattern in the IF-IR landscape: why general-purpose embedding models (EmbeddingGemma-300M, Qwen3-Embedding-0.6B) underperform on instruction-following benchmarks despite their scale, and why simply adding more instruction data to a general-purpose training mix doesn't reliably improve IF performance. The diagnostic concept is that there exists a fundamental tension between instruction supervision and data diversity when diversity comes from non-instruction-conditioned retrieval pairs.

The evidence for this tension is stark and appears in a pattern that the paper's experimental design is specifically constructed to reveal. Compare Ins-orig (480k instruct-only samples, no non-instruct data) to All-orig (440k instruct + 440k non-instruct). Intuitively, All-orig has more data (880k vs. 480k) and more diversity (it includes the non-instruct pairs from which the instruct samples were originally synthesized). One would expect general retrieval quality to improve (reasonable) and IF performance to either stay the same or improve slightly (since the instruct data is still present). Instead, IF performance catastrophically degrades: FollowIR p-MRR drops from 5.21 to 5.27 (flat, Table 1), but InfoSearch p-MRR plummets from +4.06 and +2.06 (Ins-orig) to −23.22 and −49.65 (All-orig). Negative p-MRR means the model is actively contradicting instruction-defined relevance — it's not ignoring instructions, it's doing the opposite of what they specify. And this pattern replicates on a completely different backbone (bge-m3-retromae, Table 2): InfoSearch keyword p-MRR goes from +4.18 to −62.04 under the same data mixing.

What makes this a genuine innovation rather than just an empirical observation is the mechanistic interpretation the paper offers in Section 5: "Non-instruct samples provide gradient signal that rewards query-correlated retrieval regardless of instructions; at a 50/50 mix, this overwhelms the instruction signal." This is not a claim about data quality — it's a claim about gradient conflict. The non-instruct data teaches the model a simpler, more statistically dominant pattern: when you see a query about topic X, retrieve documents about topic X. The instruct data teaches a more complex pattern: when you see a query about topic X and an instruction specifying constraint Y, retrieve documents that satisfy both X and Y. In gradient space, these two objectives conflict because the non-instruct signal pushes all documents topically related to X to have high similarity with the query representation, while the instruct signal pushes some of those same documents to have low similarity when the instruction's constraints aren't met. At a 50/50 mixing ratio, the non-instruct signal dominates — not because it's inherently stronger per sample, but because it's a simpler pattern that the model can learn more quickly and because it provides gradient updates that directly oppose the instruction-conditioned discrimination the instruct data is trying to teach.

This diagnostic concept has significant implications beyond this paper. It explains why "just scale up" doesn't work for IF-IR: scaling with non-instruct data actually makes the problem worse by strengthening the instruction-agnostic retrieval prior. It reframes instruction-following not as a capability that accumulates with scale but as "a fragile property requiring consistent supervision" (Section 5). And it provides a clear criterion for evaluating training data strategies: any method that claims to improve IF must either (a) avoid gradient conflict with non-instruct data, (b) overcome it through stronger per-sample instruction signal, or (c) provide volume without introducing non-instruction-conditioned samples. The paper's dual-view synthesis succeeds by route (c): All-DV achieves the same 880k volume as All-orig but with zero non-instruct samples, and consequently achieves both the highest p-MRR (8.30) and competitive general retrieval quality (Score 21.38), demonstrating volume without conflict.

Innovation 3: Data Volume as the Primary Driver of General Retrieval Quality, Decoupled from Source Heterogeneity

A subtler but equally important finding emerges from the comparison between All-DV and All-orig. Both have the same total training volume (880k samples) and the same number of original instruct samples (440k). The only difference is what fills the remaining 440k slots: dual-view synthetic instructions (All-DV) versus non-instruct retrieval pairs (All-orig). The non-instruct pairs provide genuine source heterogeneity — they come from a different data distribution, with different query-document pairings and no instruction conditioning. The dual-view pairs recycle the same queries and documents as the original instruct data, just with different instructions. One would intuitively expect the non-instruct data to provide more diversity and therefore better general retrieval quality.

The results contradict this intuition. All-DV achieves a FollowIR Score of 21.38 versus All-orig's 20.85 (Table 1) — the dual-view model actually has better general retrieval quality despite having less source heterogeneity. This pattern is even clearer on the bge-m3 backbone (Table 2), where All-DV achieves 20.99 versus All-orig's 20.69. And critically, All-DV achieves this better general quality while also being dramatically better at IF (p-MRR 8.30 vs. 5.27) — it wins on both dimensions simultaneously.

The paper's interpretation of this finding (Section 4, "The role of data diversity") is the key conceptual move: "This suggests that data volume, rather than source heterogeneity per se, is the primary driver of general retrieval quality, as long as individual training samples are sufficiently diverse in their query-document pairings." The italicized qualification is important, but it's the decoupling of volume from heterogeneity that is novel. The standard assumption in data-centric ML is that more diverse data sources improve generalization — you want data from different domains, different styles, different distributions to prevent overfitting and improve robustness. But in the context of retrieval training, where the task is fundamentally about learning to map queries to relevant documents in a shared embedding space, what matters most for generalization may simply be the number of distinct query-document pairings the model sees, regardless of whether those pairings come from one source or many.

If this finding generalizes — and the cross-backbone replication in Table 2 provides some evidence that it might — it has direct implications for how retrieval models should be trained. Rather than collecting diverse data sources and mixing them, practitioners might get equivalent or better results by simply generating more instruction-conditioned variants of their existing data, as the dual-view synthesis does. This is a data efficiency insight: it's better to have 880k instruction-conditioned training instances (even if they're built from only 440k underlying document sets) than 440k instruction + 440k non-instruction instances, because the former provides both volume (for general quality) and instruction signal (for IF) without gradient conflict. The dual-view synthesis resolves the data mixing tension not by finding a clever way to mix heterogeneous data without conflict, but by showing that heterogeneous data isn't necessary in the first place — volume from instruction-conditioned sources is sufficient.

5. Experimental Analysis

Evaluation Methodology

Dataset. All experiments use the Promptriever training dataset (Weller et al., 2025b) as the seed data source for both original instruction-following samples and for selecting instances eligible for polarity reversal synthesis. The training configurations draw different subsets at different sizes: Ins- configurations use 480k instruct samples, All- configurations use 440k instruct samples plus supplementary data (Section 3, "Backbone Models"). Evaluation is conducted on three benchmark suites: FollowIR (Weller et al., 2025a), InfoSearch (Zhou et al., 2025) using its length and keyword subsets, and MAIR (Sun et al., 2024) using its IFEval (Zhou et al., 2023) and InstructIR (Oh et al., 2024) subsets. No explicit train/validation/test split is described beyond the standard benchmark evaluation protocols — the paper uses the published test sets of each benchmark.
Base model(s). The primary encoder is gte-multilingual-mlm-base (Zhang et al., 2024), a 305M-parameter bi-encoder initialized from the authors' own contrastively pretrained checkpoint trained on 1.41 billion unsupervised query-document pairs from C4 (Raffel et al., 2020), mC4 (Habernal et al., 2016), CC News, and multilingual Wikipedia. To assess cross-backbone generalizability, the paper additionally trains bge-m3-retromae (Chen et al., 2024), a stronger encoder with a different pretraining strategy (retroMAE rather than standard masked language modeling). The choice of a 305M-parameter scale is deliberate: it falls in the range of general-purpose embedding models like EmbeddingGemma-300M and Qwen3-Embedding-0.6B, enabling direct comparison, but is small enough that training multiple configurations is computationally feasible.
Metrics. The paper uses three metric families. p-MRR (paired mean reciprocal rank), the central IF-sensitivity metric from Weller et al. (2025a), compares a model's ranking under two paired instructions where a document is annotated as relevant under one and not the other; positive p-MRR indicates the model correctly adjusts rankings in response to instruction changes, while negative p-MRR indicates the model ranks documents in the opposite direction from what the instruction specifies. Score on FollowIR is the macro-average across three FollowIR subsets (MAP@1000 on two subsets, nDCG@5 on one), measuring general retrieval quality independent of instruction sensitivity. nDCG@10 on MAIR subsets measures ranking quality with a focus on top-10 results, standard for retrieval evaluation. InfoSearch results are reported as p-MRR only (length and keyword subsets), consistent with that benchmark's focus on instruction sensitivity.
Baselines. The paper's primary baselines are its own training data configurations rather than external models. Ins-orig uses 480k original instruct samples from Promptriever, representing the state-of-the-art IF-IR training approach from Weller et al. (2025b). All-orig uses 440k instruct + 440k non-instruct samples from Promptriever, representing the natural data mixing strategy of supplementing with the non-instruct pairs from which the instruct samples were synthesized. External comparison points in Table 1 include EmbeddingGemma-300M (Vera et al., 2025) and Qwen3-Embedding-0.6B (Zhang et al., 2025), both general-purpose embedding models of comparable or larger scale than the primary encoder, included to demonstrate that scale alone does not solve IF-IR.
Generation budget / compute accounting. The paper's compute accounting operates at the level of training data volume, measured in number of training samples. All comparisons are size-matched: Ins-DV substitutes 240k dual-view synthetic samples for 240k original instruct samples, keeping total volume at 480k to match Ins-orig; All-DV substitutes 440k dual-view samples for 440k non-instruct samples, keeping total volume at 880k to match All-orig. Training is conducted using the Arctic-Embed framework with identical hyperparameters across all configurations: InfoNCE loss with temperature τ = 0.02, 30 hard negatives per query (including 1–3 instruction negatives), maximum sequence length 512 tokens, and query+instruction concatenation before encoding. The paper does not report training duration, GPU hours, or inference FLOPs, treating data volume as the primary compute currency for fair comparison. The one-time LLM synthesis cost (Qwen3-Next-80B-A3B-Instruct inference for generating complementary instructions) is not quantified but is described as an offline, non-recurring cost.
Cross-validation / statistical protocol. The paper does not employ cross-validation or report confidence intervals. The only quality control reported is a manual inspection of 100 synthesized instructions, confirming over 99% usability with no additional filtering applied. All results in Tables 1 and 2 are point estimates from single training runs. The paper does not report variance across random seeds or training runs, making it impossible to assess whether the observed differences between configurations (e.g., FollowIR Score dropping from 21.33 to 19.73 from Ins-orig to Ins-DV) are statistically significant or within training noise.

Main Quantitative Results

Instruct-Only Comparison: Dual-View Data Improves IF But May Reduce General Retrieval Quality

The Ins- configurations compare Ins-DV (240k original + 240k dual-view) against Ins-orig (480k original) at the same total data volume of approximately 480k samples. This comparison isolates the effect of replacing half the original instruct samples with their dual-view counterparts while holding data volume constant.

Headline result on instruction-following sensitivity. Across all IF-specific metrics, Ins-DV substantially outperforms Ins-orig. FollowIR p-MRR increases from 5.21 to 7.57 (+45%, Table 1), which the paper highlights as surpassing general-purpose models EmbeddingGemma-300M (5.61) and Qwen3-Embedding-0.6B (5.09). InfoSearch length p-MRR improves from 4.06 to 9.02 (+122%), and InfoSearch keyword p-MRR improves from 2.06 to 5.61 (+172%). MAIR IFEval nDCG@10 rises from 32.14 to 36.13. These gains are consistent in direction and substantial in magnitude: the smallest relative improvement across IF metrics is +12.4% on IFEval; the largest is +172% on InfoSearch keyword. On the bge-m3-retromae backbone (Table 2), the pattern replicates: FollowIR p-MRR increases from 9.40 to 11.47 (+22%), InfoSearch length p-MRR from 19.00 to 28.64 (+51%), and InfoSearch keyword p-MRR from 4.18 to 48.42 — a striking 11.6× improvement on the keyword subset.

Trade-off on general retrieval quality. The FollowIR Score — which measures general retrieval quality independent of instruction sensitivity — drops from 21.33 (Ins-orig) to 19.73 (Ins-DV) on the primary backbone (Table 1). The same pattern appears on bge-m3 (Table 2): Score drops from 22.26 to 19.76. The paper attributes this to reduced training context diversity: since each dual-view pair reuses the same query and documents as the original it replaces (only the instruction changes), the model sees fewer distinct query-document pairings when half the original samples are substituted. On MAIR, the pattern is mixed: InstructIR nDCG@10 drops from 89.16 to 87.97 (primary backbone, Table 1) and from 90.40 to 89.53 (bge-m3, Table 2), consistent with the diversity explanation. But IFEval nDCG@10 increases from 32.14 to 36.13 (primary) and from 27.12 to 29.37 (bge-m3), showing that on some general-retrieval subsets, the dual-view signal may provide a net positive even for non-instruction-sensitivity metrics — possibly because IFEval inherently requires some instruction-following capability to achieve high scores.

Interpretation. The Ins- comparison demonstrates that per training sample, dual-view data provides stronger instruction supervision than original instruct data — the model learns more IF capability from 240k original + 240k DV than from 480k original alone. But this comes at a cost to general retrieval quality because the substitution reduces the diversity of distinct query-document pairs in the training set. This trade-off is the first piece of the data mixing tension the paper diagnoses.

Mixed-Data Comparison: Dual-View Data Resolves the Data Mixing Catastrophe

The All- configurations compare two strategies for scaling the training set to approximately 880k samples while holding the original instruct portion fixed at 440k: supplementing with non-instruct data (All-orig) versus supplementing with dual-view synthetic data (All-DV). This comparison isolates the effect of what type of supplementary data is used, holding both total volume and original instruct count constant.

Headline result on the data mixing catastrophe. All-orig, which supplements 440k instruct samples with 440k non-instruct samples, shows catastrophic degradation on IF sensitivity metrics relative to the instruct-only Ins-orig baseline. On InfoSearch, this degradation is extreme: length p-MRR plummets from +4.06 (Ins-orig) to −23.22 (All-orig, Table 1), and keyword p-MRR plummets from +2.06 to −49.65. Negative p-MRR values indicate the model is actively contradicting instruction-defined relevance — it is not merely ignoring instructions but systematically ranking documents in the opposite direction from what the instruction specifies. On FollowIR, p-MRR is essentially flat (5.21 to 5.27), but this stability masks the severity of the InfoSearch collapse. On MAIR, IFEval drops from 32.14 to 24.33 and InstructIR drops from 89.16 to 85.54. The bge-m3 backbone (Table 2) shows the same pattern with even more extreme magnitudes: InfoSearch keyword p-MRR crashes from +4.18 to −62.04, and length p-MRR from 19.00 to −27.08. The data mixing catastrophe is not backbone-specific — it is a robust phenomenon.

Critically, this catastrophic degradation occurs despite the fact that All-orig contains more total data (880k vs. 480k) and the same instruction data that drives IF capability. The non-instruct samples do not remove the instruct signal — they overwhelm it in gradient space.

Dual-view data reverses the degradation. All-DV, which uses 440k dual-view synthetic samples instead of 440k non-instruct samples (same total volume, same original instruct count, same backbone), achieves the best results across nearly all metrics. FollowIR p-MRR reaches 8.30 (Table 1), the highest among all configurations and surpassing Ins-DV (7.57) — the additional volume from DV samples provides further IF improvement beyond what the Ins-DV substitution achieved. InfoSearch length p-MRR jumps to +31.91 (vs. −23.22 for All-orig, a swing of +55.13) and keyword p-MRR to +12.13 (vs. −49.65, a swing of +61.78). These are not marginal improvements but qualitative reversals: the model goes from actively contradicting instructions to strongly following them. FollowIR Score reaches 21.38, the highest among all configurations and slightly above Ins-orig (21.33), indicating that general retrieval quality is fully recovered and slightly improved relative to the instruct-only baseline. On MAIR, IFEval reaches 34.08 (vs. 24.33 for All-orig and 32.14 for Ins-orig) and InstructIR hits 90.74 (the highest across all configurations). On bge-m3 (Table 2), the pattern is identical: FollowIR p-MRR reaches 13.92 (highest), InfoSearch length p-MRR reaches 40.15 (vs. −27.08), keyword p-MRR 49.62 (vs. −62.04), with competitive Score (20.99 vs. 20.69 for All-orig).

The data volume explanation. The paper's key interpretive claim is that All-DV achieves this dual improvement — best IF and best general retrieval — because it provides volume without gradient conflict. The 440k DV samples increase the total number of training instances (from 480k to 880k, comparing Ins-DV to All-DV), which sustains general retrieval quality through data volume alone, without introducing non-instruction-conditioned samples that would dilute the instruction signal. The fact that All-DV contains no non-instruct data yet achieves the best FollowIR Score (21.38, Table 1) supports the paper's claim that "data volume, rather than source heterogeneity per se, is the primary driver of general retrieval quality" — the DV samples, despite recycling the same underlying query-document sets as the original instruct data, provide sufficient volume to match or exceed the general retrieval quality of configurations with genuine source heterogeneity (All-orig's 20.85, Table 1).

Comparison to external baselines. The paper contextualizes these results against general-purpose embedding models. EmbeddingGemma-300M achieves 5.61 FollowIR p-MRR and Qwen3-Embedding-0.6B achieves 5.09 (Table 1). All-DV's 8.30 substantially exceeds both despite using a 305M-parameter backbone comparable in scale to these models. This demonstrates that the IF-IR gap is not a capacity problem — models of this scale can achieve strong IF performance with the right training data. The bge-m3 results in Table 2 (13.92 p-MRR) further suggest that a stronger pretrained backbone amplifies the benefit of the DV signal rather than diminishing it.

Cross-Backbone Generalizability

Table 2 replicates the full experimental design on bge-m3-retromae, a stronger encoder with a fundamentally different pretraining strategy (retroMAE rather than standard masked language modeling). All qualitative patterns from Table 1 replicate:

Ins-DV improves IF over Ins-orig: FollowIR p-MRR 11.47 vs. 9.40 (+22%), InfoSearch keyword p-MRR 48.42 vs. 4.18 (11.6× improvement). The magnitude of the keyword improvement is substantially larger on bge-m3 than on the primary backbone (11.6× vs. 2.7×), suggesting that a stronger backbone amplifies the DV signal's effectiveness, possibly because the retroMAE pretraining produces representations that are more amenable to instruction-conditioned fine-tuning.
Ins-DV sacrifices general retrieval quality: FollowIR Score drops from 22.26 to 19.76, mirroring the primary backbone's pattern.
All-orig catastrophically degrades IF: InfoSearch keyword p-MRR crashes to −62.04, even worse than on the primary backbone (−49.65), and length p-MRR to −27.08.
All-DV reverses the degradation and achieves the best IF: FollowIR p-MRR 13.92 (highest overall), InfoSearch length p-MRR 40.15, keyword p-MRR 49.62, with competitive Score (20.99).

The cross-backbone replication is methodologically important because it rules out the possibility that the data mixing tension or the DV benefit are artifacts of the specific pretrained encoder. Both phenomena are robust to backbone architecture and pretraining strategy, supporting the paper's claim that they are fundamental properties of contrastive retrieval training with instruction data, not implementation details.

Ablation Studies and Robustness Checks

The paper does not report traditional ablation studies in the sense of removing components of the proposed method and measuring degradation. What it provides instead is a controlled data composition design that functions as a series of implicit ablations. Each comparison between configurations isolates a specific data factor:

Ins-DV vs. Ins-orig ablates the effect of replacing original instruct data with dual-view data at fixed volume, revealing the per-sample IF supervision benefit and the diversity cost.
All-orig vs. Ins-orig ablates the effect of adding non-instruct data to an instruct-only training set, revealing the data mixing catastrophe.
All-DV vs. All-orig ablates the effect of dual-view data vs. non-instruct data as the supplementary volume source, revealing that DV resolves the mixing tension.
All-DV vs. Ins-DV implicitly ablates the effect of scaling instruction-conditioned data volume (880k vs. 480k), showing that additional DV samples provide further IF improvement (p-MRR 8.30 vs. 7.57) and recover general retrieval quality (Score 21.38 vs. 19.73), though this comparison confounds volume and DV-vs-original composition.

Effect of instruction negative presence in seed data. The paper implicitly ablates the requirement for instruction negatives by only generating DV samples from instances that already contain them. Instances without instruction negatives are not augmented — they remain as original instruct-only samples in all configurations. The paper does not analyze whether the quality or quantity of instruction negatives per instance affects the effectiveness of the synthesized complementary instruction, nor does it compare performance when DV synthesis is applied to instances with multiple instruction negatives versus only one.

Effect of LLM-based synthesis quality. The 99%+ usability rate from manual inspection is reported but not experimentally interrogated. The paper does not report an ablation where deliberately corrupted instructions (e.g., random instructions, irrelevant instructions, trivial rewordings) are used as dual-view pairs to measure the performance cost of synthesis failures. Without such a control, it is difficult to assess whether the observed gains come specifically from the polarity-reversal property or more generally from any form of instruction augmentation.

Effect of temperature and negative count. All configurations use the same training hyperparameters (τ = 0.02, 30 hard negatives). The paper does not ablate whether the benefit of dual-view training is sensitive to temperature (lower temperatures might amplify the DV signal by making the contrastive loss more sensitive to small differences) or to the number of instruction negatives per query (more instruction negatives might provide more opportunities for reversal synthesis).

Negative result: non-instruct data mixing degrades IF, and this is robust. The catastrophic degradation from non-instruct mixing is observed in both All-orig configurations across both backbones (Tables 1 and 2), making it the most robust and most replicated finding in the paper. The fact that the degradation is qualitatively severe (sign flips from positive to large negative p-MRR) rather than a marginal decline suggests it is not a statistical fluke. However, without multiple training runs or confidence intervals, the exact magnitude of the degradation is uncertain, and the paper does not explore whether the effect is monotonic — for example, whether a 25/75 non-instruct/instruct mix would avoid the catastrophe while still providing some diversity benefit, or whether any non-instruct contamination is harmful.

Critical Assessment

The experiments demonstrate a set of internally consistent patterns that support the paper's core narrative: dual-view synthesis improves IF sensitivity, non-instruct data mixing degrades it catastrophically, and dual-view synthesis resolves this tension by providing volume without gradient conflict. However, there are significant limitations in what the experiments demonstrate versus what the paper claims, and several important questions are left unanswered.

Claim 1: Dual-view synthesis improves FollowIR p-MRR by 45%.

The experimental evidence for this specific claim is clear and well-supported: on the primary backbone, Ins-DV achieves 7.57 p-MRR versus 5.21 for Ins-orig at the same data volume (Table 1), a 45.3% relative improvement. This comparison is clean — same model, same training protocol, same total data volume, same proportion of original instruct samples (the 50% substitution means half the original samples are replaced with their DV counterparts). The improvement is replicated on bge-m3 (9.40 to 11.47, +22% relative improvement, Table 2).

However, two qualifications are necessary. First, the 45% figure applies specifically to the Ins-DV configuration (instruct-only, size-matched substitution), not to the paper's best overall result (All-DV's 8.30 p-MRR). The All-DV configuration uses a different instruct count (440k vs. 480k) and a different volume (880k vs. 480k), making the Ins-DV comparison the proper reference for the 45% claim. But the 45% improvement is measured relative to a baseline (Ins-orig) that the paper subsequently shows is suboptimal — the alignment between the specific comparison and the paper's broader narrative about the method's effectiveness holds, but readers should note that the absolute p-MRR numbers are small (single digits) and the percentage improvement, while large in relative terms, represents a shift from "barely above random" to "modestly above random" in absolute terms.

Second, the 45% improvement on p-MRR is not the full picture. The same configuration (Ins-DV) shows a 7.5% decrease in FollowIR Score (21.33 to 19.73), which measures general retrieval quality. The paper acknowledges this trade-off and frames it as part of the data mixing tension it identifies, but the headline claim of "45% improvement" should be understood as applying to instruction sensitivity specifically, not to overall retrieval quality.

Claim 2: Dual-view synthesis resolves the tension between data diversity and instruction supervision.

The evidence for this claim comes from the comparison between All-DV and All-orig at matched volume (880k). All-DV achieves both better IF (p-MRR 8.30 vs. 5.27) and better general retrieval quality (Score 21.38 vs. 20.85) than All-orig on the primary backbone (Table 1). This is a genuine improvement on both axes simultaneously, and it is replicated on bge-m3 (Table 2), where All-DV achieves p-MRR 13.92 vs. 8.84 and Score 20.99 vs. 20.69.

The critical interpretive question is why this resolution works. The paper claims it works because "data volume, rather than source heterogeneity per se, is the primary driver of general retrieval quality" — the DV samples provide volume without gradient conflict. This interpretation is supported by the observation that All-DV (which has less source heterogeneity than All-orig, since its 440k supplementary samples recycle the same document sets as the original instruct data) achieves better general retrieval quality. If source heterogeneity were the primary driver, All-orig would outperform on Score, but it doesn't.

However, there is an alternative interpretation the paper does not rule out: All-DV might achieve better general retrieval not because volume is sufficient, but because the DV samples provide a different kind of diversity that happens to be beneficial — specifically, they force the model to learn more robust query-document associations by presenting the same document under different relevance labels, which may prevent overfitting to fixed query-document pairs in a way that non-instruct data doesn't. Under this interpretation, the DV samples don't just provide volume; they provide a qualitatively different and superior form of diversity that non-instruct data lacks. The paper's claim that volume alone drives general retrieval quality is asserted rather than demonstrated — a configuration with additional original instruct samples (not DV, just more distinct instruct triplets from Promptriever) at the same 880k volume would be needed to test whether volume per se or the specific properties of DV data drive the Score recovery. If 440k original instruct + 440k original instruct (from a disjoint subset of Promptriever) achieved similar Score to All-DV, the volume interpretation would be supported. If it did not, the DV-specific properties would be the driver. This ablation is not reported.

Claim 3: The data mixing tension is a fundamental phenomenon, with non-instruct data degrading IF.

This is the most robustly demonstrated claim in the paper. The catastrophic degradation of InfoSearch p-MRR from positive to large negative values when non-instruct data is mixed in (All-orig vs. Ins-orig) appears on both backbones (Tables 1 and 2) and is consistent in direction across all IF metrics. The sign flip from positive to negative p-MRR is particularly compelling because it cannot be explained by mere dilution of the instruction signal — dilution would push p-MRR toward zero (random behavior), not toward large negative values. The negative p-MRR implies that the model is actively learning the opposite of instruction-following from the mixed data, which is consistent with the paper's gradient conflict hypothesis: non-instruct samples push the model to retrieve documents based on topical similarity alone, and at a 50/50 mix, this simpler signal dominates the instruction-conditioned signal, causing the model to systematically rank documents that satisfy the instruction lower than documents that violate it.

However, there are important limitations to what has been demonstrated. The paper tests exactly one mixing ratio: 50/50 instruct/non-instruct (All-orig, 440k + 440k). It does not report results for other ratios (e.g., 25/75, 75/25, 90/10) that would characterize the shape of the degradation curve. It is possible that a small amount of non-instruct data (say, 10%) provides diversity benefits without causing catastrophic IF degradation, and that an optimal mixing ratio exists. The paper's framing suggests that any non-instruct contamination is harmful, but the experiments only demonstrate harm at 50/50. Similarly, the paper does not test whether the degradation can be mitigated through training strategies like curriculum learning (train on instruct-only first, then mix), loss weighting (up-weight instruct samples), or gradient surgery techniques that would separate conflicting gradient signals.

What is missing: statistical rigor, scale testing, and mechanism verification.

The paper reports point estimates from single training runs with no confidence intervals, standard deviations, or significance tests. The differences between configurations on some metrics are small: FollowIR Score for Ins-DV (19.73) vs. Ins-orig (21.33) represents a 1.6-point difference on a metric whose scale and variance are not characterized. Without knowing the run-to-run variance, it is impossible to assess whether this is a genuine diversity cost or noise. Similarly, All-DV's Score advantage over All-orig (21.38 vs. 20.85, a 0.53-point difference) could easily fall within training variance. The InfoSearch p-MRR swings are large enough (±50+ points) that they are almost certainly real effects, but the exact magnitudes are unstable without replication.

The paper operates at a single model scale (305M parameters for the primary backbone) and with a single class of architectures (bi-encoders). While the cross-backbone replication with bge-m3 provides some evidence of generality, both backbones are transformer-based bi-encoders of similar scale (bge-m3-retromae is also in the hundreds-of-millions parameter range). The paper's claims about instruction sensitivity not being a capacity issue are supported (EmbeddingGemma-300M and Qwen3-Embedding-0.6B underperform despite similar or larger scale), but the paper does not test whether the DV benefit scales with model size — would a 1B+ parameter encoder show the same 45% relative improvement, or would the benefit saturate or even diminish as the model has more capacity to learn instruction distinctions from standard data? The bge-m3 results showing a larger absolute benefit on some metrics (keyword p-MRR improvement of 44.24 points vs. 3.55 points on the primary backbone) hint that stronger backbones may benefit more, but this is suggestive rather than systematic.

Crucially, the paper does not provide direct evidence for the gradient conflict mechanism it invokes to explain the data mixing tension (Section 5). The mechanism is inferred from the pattern of results — non-instruct mixing causes IF degradation, dual-view data doesn't, and dual-view data contains no non-instruct samples — but no gradient-level analysis is performed. Measuring the cosine similarity between gradients from instruct and non-instruct samples during training, or tracking how query-instruction representations diverge over the course of training, would provide direct evidence for or against the gradient conflict hypothesis. The paper's mechanistic claims, while plausible and consistent with the observed results, remain interpretive rather than demonstrated.

What experiments would strengthen the paper.

Several missing experiments would significantly strengthen the claims:

A volume-matched original instruct baseline for All-DV: Compare All-DV (440k original + 440k DV) against a configuration with 880k original instruct samples (no DV, just more distinct Promptriever triplets). This would test whether the Score recovery in All-DV is due to volume per se or to DV-specific properties. The paper claims volume is sufficient; this experiment would provide direct evidence.
Mixing ratio sweeps: Test All configurations with 90/10, 75/25, 50/50, 25/75, 10/90 instruct/non-instruct ratios to characterize the degradation curve and identify whether a safe mixing regime exists. The paper's current binary comparison (pure instruct vs. 50/50 mix) may overstate the severity of the mixing problem.
Gradient analysis: Compute and compare the gradient cosine similarity between instruct-only and non-instruct-only training steps during training to directly verify gradient conflict. Track the evolution of query-instruction representation distances under different data configurations.
Loss weighting or curriculum learning as baselines: If gradient conflict is the mechanism, methods that mitigate gradient conflict (loss weighting, gradient projection, curriculum learning starting with instruct-only and gradually introducing non-instruct data) should help. Testing these would both validate the mechanism and provide practical alternatives to DV synthesis.
Synthesis quality ablation: Compare DV configuration where 10%, 25%, or 50% of the synthesized instructions are deliberately corrupted (random, irrelevant, or trivial paraphrases) against the 99%+ quality configuration to measure sensitivity to synthesis quality and establish the importance of genuine polarity reversal versus merely having additional instruction text.
Statistical characterization: Report standard deviations or confidence intervals from multiple training runs (at least 3–5 seeds) for the key comparisons to establish which differences are statistically reliable. The FollowIR Score differences between Ins-DV and Ins-orig, and between All-DV and All-orig, are small enough that single-run estimates are insufficient.

6. Limitations and Trade-offs

Practical Deployment Cost of LLM-Based Instruction Synthesis Is Unaccounted For

The assumption or constraint. The paper presents dual-view synthesis as requiring "no additional annotation cost" (Section 6) and being deployable "with no changes to existing pipelines." However, the synthesis pipeline requires running inference with Qwen3-Next-80B-A3B-Instruct — an 80B-parameter mixture-of-experts model — on every eligible training instance to generate complementary instructions. The paper states that DV samples "are generated once via LLM prompting and then used as standard training data" but never quantifies the computational cost of this generation step relative to the downstream training cost or the benefit obtained. For the All-DV configuration, this means generating approximately 440k complementary instructions by prompting an 80B-parameter model, each requiring a detailed reasoning trace before producing the final instruction output (Appendix A shows the prompt includes mandated step-by-step reasoning).

The consequence. The headline claim — that the approach has "no additional annotation cost" — conflates annotation cost (human labeling expense) with total cost (which includes LLM inference). For an organization deciding whether to adopt this method, the synthesis cost is a real line item: 440k inferences through an 80B MoE model is a substantial compute expenditure, potentially comparable to or exceeding the cost of training the 305M-parameter encoder itself. The paper does not amortize this one-time synthesis cost into its efficiency claims or provide a break-even analysis — how much IF improvement is needed to justify the synthesis compute? If the synthesis cost is comparable to simply collecting more original instruct data or training a larger model, the practical advantage of the method diminishes. Furthermore, the synthesis quality depends on the specific LLM used (Qwen3-Next-80B-A3B-Instruct); organizations without access to this model may obtain different-quality complementary instructions, and the paper provides no guidance on minimum LLM capability requirements for successful synthesis.

What evidence exists in the paper. None. The paper does not report the number of LLM inference calls needed, the tokens generated per call, the GPU hours consumed, or the wall-clock time for synthesizing the 240k or 440k DV samples. The synthesis step is treated as an offline, non-recurring cost whose magnitude is simply not discussed. Section 3 mentions the synthesizer once ("We employ Qwen3-Next-80B-A3B-Instruct as the backbone LLM for our data synthesis pipeline") and provides the prompt template (Appendix A), but contains no cost characterization whatsoever.

Mitigation status. Not addressed. The paper frames cost solely in terms of annotation expense (human labeling not required) and pipeline changes (none needed), but does not acknowledge, measure, or attempt to reduce the LLM inference cost. Future work directions (Section 6) do not mention synthesis cost reduction, using smaller LLMs for generation, or amortization analysis.

Generalization Is Restricted to a Single Task Family (Retrieval), a Single Language (English), and Bi-Encoder Architectures

The assumption or constraint. All experiments are conducted exclusively on English-language retrieval benchmarks (FollowIR, InfoSearch, MAIR) using bi-encoder architectures (gte-multilingual-mlm-base and bge-m3-retromae). The paper acknowledges this explicitly in its Limitations section: "We evaluate on encoder-based bi-encoder retrievers; exploring the applicability to decoder-based or cross-encoder architectures is a natural direction for future work. Additionally, our experiments focus on English-language benchmarks, and extending the approach to multilingual settings remains an interesting avenue to explore." The method is designed for tasks where relevance is defined by a document satisfying instruction constraints, and where training data consists of query-document pairs with instruction negatives — a structure specific to instruction-following retrieval and not directly transferable to, for example, instruction-following in text generation, dialogue, or multimodal retrieval.

The consequence. The paper's central claims — that polarity reversal improves instruction sensitivity by 45%, that data mixing causes catastrophic IF degradation, and that DV synthesis resolves this tension — are demonstrated only within this narrow scope. The method's effectiveness on cross-encoder architectures (where query-document interaction is modeled jointly rather than through separate embeddings) is unknown and potentially different: cross-encoders can condition document representations on the query-instruction at the token level, which might make them inherently more instruction-sensitive and reduce the marginal benefit of dual-view training. For decoder-based retrieval (where a generative model produces document identifiers or relevance judgments), the contrastive training dynamic that the paper relies on does not directly apply, and whether polarity reversal provides benefit in a likelihood-based training regime is an open question. The restriction to English means the method's effectiveness is untested for languages with different syntactic structures, morphological complexity, or data availability — the LLM synthesizer (Qwen3-Next-80B-A3B-Instruct) was evaluated on English data only, and its ability to generate coherent complementary instructions in other languages is unknown.

What evidence exists in the paper. The cross-backbone replication with bge-m3-retromae (Table 2) provides evidence that the method is not specific to one pretraining strategy (retroMAE vs. standard MLM), but both backbones are transformer-based bi-encoders operating on English text. The paper explicitly notes the language and architecture scope in the Limitations section but provides no experiments or analysis that suggest these findings would transfer. The information retrieval task family is inherent to all three evaluation benchmarks — there is no evidence about whether dual-view data synthesis would help or hurt on non-retrieval instruction-following tasks.

Mitigation status. Partially. The authors acknowledge these as explicit limitations and frame them as future work. The cross-backbone replication provides weak evidence of generalization within the bi-encoder class, but no experiments address cross-architecture, cross-task, or cross-language transfer. A practitioner deploying IF-IR in a multilingual setting or with a cross-encoder architecture receives no empirical guidance from this paper.

Difficulty Estimation and Instruction-Negative Quality Requirements Are Not Characterized

The assumption or constraint. The polarity reversal mechanism assumes that for each training instance with an instruction negative, a meaningful complementary instruction exists that properly reverses relevance labels. Specifically, an instruction negative must be genuinely "relevant to the pure query" (Section 2) — a document that differs from the positive along instruction-relevant attribute dimensions rather than being fundamentally off-topic. If the instruction negative is a poor-quality sample (e.g., only marginally topically relevant, or relevant-but-excluded for a trivial reason), the synthesized complementary instruction may be nonsensical, overly narrow, or fail to meaningfully contrast with the original instruction. The paper reports 99%+ usability from a 100-sample manual check but provides no characterization of what "usable" means, what the failure modes look like for the <1% of unusable samples, or whether the quality of the original instruction negatives correlates with downstream DV training benefit.

The consequence. For practitioners, the method's effectiveness depends on an input data property — the quality and nature of existing instruction negatives — that the paper does not help them assess. If a practitioner's training data has instruction negatives that are weaker (e.g., only marginally different from positives, or excluded for superficial reasons like word count rather than substantive attribute differences), the synthesized complementary instructions may produce trivial reversals (e.g., "include short documents" vs. "include long documents") that teach the model to attend to superficial features rather than genuine instructional content. The paper's volcano example (Figure 1) shows a substantive reversal involving physical formation vs. environmental impacts — but it's unclear what fraction of the DV training data contains similarly rich reversals versus trivial ones. Without this characterization, the method's expected benefit on a new dataset is uncertain.

Furthermore, the synthesis pipeline's reliability likely depends on the LLM's ability to identify distinguishing attributes between documents and formulate constraints targeting those attributes. The paper reports 99%+ usability with Qwen3-Next-80B-A3B-Instruct, but this is measured by one annotator on 100 samples from one dataset. Whether this quality rate holds for different LLM synthesizers, different data distributions (e.g., technical legal or medical documents with more subtle attribute distinctions), or for instruction negatives identified by different methods (e.g., automated hard negative mining rather than Promptriever's construction) is unknown.

What evidence exists in the paper. Minimal. The 99%+ usability rate is the only quality metric reported, and it is based on a single annotator checking 100 datapoints (Section 2). The paper does not report inter-annotator agreement, the criteria used for "usable," example failure cases, or correlation between instruction negative properties (e.g., topical similarity to the positive, length, attribute distinctiveness) and downstream performance. Appendix A provides the prompt template, which reveals the complexity of the synthesis task and suggests many ways it could fail, but the paper does not analyze actual failure modes or their frequency beyond the coarse "over 99% usable" claim.

Mitigation status. Not addressed. The paper treats the 99%+ usability rate as sufficient justification for applying no additional filtering, but does not investigate whether the remaining <1% of failures cause training harm (they would introduce inconsistent relevance labels that create gradient conflict), whether different seed datasets would produce different quality rates, or what minimum instruction-negative quality is needed for the method to be effective. These are not flagged as limitations or future work.

Single Mixing Ratio Evaluated for the Data Mixing Tension, Leaving the Degradation Curve Uncharacterized

The assumption or constraint. The paper's central diagnostic finding — that mixing non-instruct data catastrophically degrades IF performance — is demonstrated at exactly one mixing ratio: 50/50 instruct/non-instruct in the All-orig configuration (440k instruct + 440k non-instruct = 880k total). The paper generalizes from this single data point to strong claims about the nature of instruction sensitivity: "Instruction sensitivity is therefore not a capability that accumulates with scale but a fragile property requiring consistent supervision" (Section 5), and "non-instruct samples provide gradient signal that rewards query-correlated retrieval regardless of instructions; at a 50/50 mix, this overwhelms the instruction signal."

The consequence. The shape of the degradation curve is unknown. It is possible that a small amount of non-instruct data (e.g., a 90/10 instruct/non-instruct mix) provides diversity benefits for general retrieval quality without causing catastrophic IF degradation — the degradation might have a threshold or nonlinear behavior that the single 50/50 data point cannot characterize. If a 90/10 mix maintains IF performance while improving general retrieval, the paper's framing that non-instruct data is fundamentally harmful to IF would be overstated, and practitioners could use mixing strategies that balance both objectives without requiring the full DV synthesis approach. Conversely, if even 10% non-instruct data causes significant degradation, the paper's strong claims would be supported, but the severity of the problem would be even more acute than demonstrated.

The paper's theoretical interpretation — gradient conflict where non-instruct signal overwhelms instruct signal — implicitly predicts that degradation should be monotonic with the proportion of non-instruct data: more non-instruct data should produce more degradation. But the negative p-MRR values observed for All-orig (InfoSearch length −23.22, keyword −49.65 in Table 1) are not just degradation — they are instruction reversal, where the model actively contradicts instructions. This qualitative shift (from positive to negative p-MRR) suggests a phase transition rather than monotonic degradation, and the mixing ratio at which this transition occurs is critical for practical deployment but completely uncharacterized.

What evidence exists in the paper. Only the single 50/50 ratio (All-orig configuration, Tables 1 and 2). The Ins-orig configuration (480k instruct, 0% non-instruct) provides the other endpoint of the mixing spectrum, but there are no intermediate data points. The paper's Section 4 describes the mixing tension as a binary phenomenon (instruct-only good, mixed bad) without investigating the continuous relationship between mixing ratio and IF performance.

Mitigation status. Not addressed. The paper does not report mixing ratio sweeps, does not discuss whether a safe mixing regime might exist, and does not flag the single-ratio evaluation as a limitation. The gradient conflict hypothesis (Section 5) is offered as explanation for the 50/50 result but is not tested at other ratios where the conflict would be expected to scale proportionally. This is a significant gap because the paper's practical recommendation — use DV synthesis instead of non-instruct data — is based on the assumption that non-instruct mixing is harmful in general, not just at 50/50.

General Retrieval Quality Metric Lacks Statistical Characterization and Shows Small, Unstable Differences

The assumption or constraint. The FollowIR Score metric — which the paper uses as its primary measure of general retrieval quality to support the claim that DV synthesis "simultaneously improves both dimensions at equal data budget" (Section 6) — shows differences between configurations that are small in magnitude: 21.38 (All-DV) vs. 20.85 (All-orig) vs. 21.33 (Ins-orig) vs. 19.73 (Ins-DV) on the primary backbone (Table 1). These represent a total range of 1.65 points across all four configurations. The paper reports these as point estimates from single training runs with no standard deviations, confidence intervals, or statistical tests, and provides no characterization of the FollowIR Score's variance across training runs or random seeds.

The consequence. The paper's central narrative — that All-DV resolves the tension between IF and general retrieval by achieving the best of both — rests partially on the claim that All-DV achieves the best FollowIR Score (21.38, Table 1). But this "best" represents a 0.05-point advantage over Ins-orig (21.33) and a 0.53-point advantage over All-orig (20.85). Without variance estimates, it is impossible to determine whether these differences are statistically significant or reflect training noise. If the metric's run-to-run standard deviation is, say, ±0.5 points, then the observed differences are within noise and the claim that All-DV achieves the best general retrieval quality is unsupported. The Ins-DV Score drop (19.73 vs. 21.33 for Ins-orig) is larger at 1.60 points, which is more likely to be a real effect, but the magnitude is still modest and its statistical reliability is unknown.

This limitation does not undermine the paper's IF-specific claims — the p-MRR improvements are large enough (Ins-DV's 7.57 vs. 5.21, a 2.36-point difference; All-DV's 8.30 vs. 5.27, a 3.03-point difference) that they are almost certainly real effects despite lacking formal tests. But it substantially weakens the claim that "data diversity" and "instruction supervision" are resolved simultaneously, because the "data diversity" half of the resolution (general retrieval quality) rests on small differences that may be unreliable. On the bge-m3 backbone (Table 2), the same pattern appears: All-DV Score 20.99 vs. All-orig 20.69 vs. Ins-orig 22.26 vs. Ins-DV 19.76. Here the All-DV vs. All-orig difference is only 0.30 points, and All-DV actually underperforms Ins-orig (20.99 vs. 22.26) — meaning the best general retrieval on bge-m3 is achieved by Ins-orig (instruct-only, no DV, no non-instruct), which contradicts the "best of both" narrative for that backbone.

What evidence exists in the paper. None regarding variance. Tables 1 and 2 report point estimates with no error characterization. The paper does not mention multiple training runs, random seeds, or any form of statistical testing anywhere in the experimental setup (Section 3) or results (Section 4). The MAIR nDCG@10 metrics, which also measure general retrieval quality, show mixed patterns: on the primary backbone, All-DV achieves 90.74 on InstructIR (best) but 34.08 on IFEval (second-best, behind Ins-DV's 36.13), further complicating the claim that All-DV dominates on general retrieval.

Mitigation status. Not addressed. The paper treats the point estimates as reliable rankings without acknowledging that the small Score differences may be within training noise. No statistical tests, confidence intervals, or multi-seed training runs are reported. The cross-backbone replication (Table 2) provides a form of independent verification — the patterns are qualitatively consistent across backbones — but since single runs are used on each backbone and the Score differences remain small, this does not constitute a statistical characterization. The paper's claim that DV synthesis "simultaneously improves both dimensions" should be treated as a suggestive pattern requiring statistical validation, not an established finding.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper makes a diagnostic and methodological contribution rather than proposing a new architecture or training objective. Its primary shift is to reframe the instruction-following retrieval problem from one of model capacity or data volume to one of data composition and gradient conflict. Before this work, the dominant narrative in IF-IR — implicit in the progression from Promptriever (Weller et al., 2025b) to InF-IR (Zhuang et al., 2025) — was that IF capability requires either dedicated instruction-conditioned training data (which helps but is expensive to create) or sufficient model scale (which should theoretically enable instruction generalization but empirically doesn't, as evidenced by EmbeddingGemma-300M and Qwen3-Embedding-0.6B achieving p-MRR scores of 5.61 and 5.09 in Table 1). This paper changes the conversation by demonstrating that the bottleneck is neither capacity nor total instruction data volume, but rather what kind of signal the training data provides about the relationship between instructions and relevance.

The methodological shift is from "more instruction data" to "richer instruction contrast." Promptriever showed that instruction negatives — documents that are topically relevant but instructionally wrong — are valuable training signals. This paper shows that instruction negatives are structurally underused when treated only as fixed failures: each one implicitly defines a complementary instruction under which it becomes relevant, and making that complement explicit through LLM synthesis creates a training instance that teaches not just what to exclude but what distinguishes one instruction from another. This is a shift from instructional exclusion (learning what each instruction rules out) to instructional distinction (learning where two instructions diverge such that they demand opposite documents). The 45% improvement in FollowIR p-MRR from Ins-DV over Ins-orig at matched data volume (5.21 to 7.57, Table 1) suggests that per training sample, polarity-reversed instruction pairs are substantially more effective at teaching instruction sensitivity than standard instruction-negative training, because they directly penalize the instruction-agnostic representations that cause IF failure.

Resolving a contradiction in the IF-IR landscape. The paper provides a unified explanation for a puzzling pattern: why do general-purpose embedding models with massive training corpora (EmbeddingGemma, Qwen3-Embedding) underperform small, instruction-specialized models on IF benchmarks, despite having comparable or larger parameter counts? And why does simply adding instruction data to a general-purpose training mix not reliably produce instruction-sensitive models? The paper's data mixing experiments (All-orig in Tables 1 and 2) answer both questions through the gradient conflict mechanism: non-instruct retrieval training teaches a dominant, simpler pattern — "retrieve documents topically related to the query" — that directly conflicts with the instruction-conditioned signal. At typical mixing ratios (50/50), this simpler signal overwhelms the instruciton signal, producing models that not only fail to follow instructions but actively contradict them (negative p-MRR on InfoSearch: −23.22 length, −49.65 keyword in Table 1). This explains why larger general-purpose models aren't better at IF — scaling with non-instruct data strengthens the instruction-agnostic prior rather than teaching instruction sensitivity. It also explains why InF-IR achieved competitive IF with only ~38k triplets (Zhuang et al., 2025): those triplets were instruction-pure, avoiding the gradient conflict that cripples mixed-data training.

This reframing has concrete implications for which research directions become more versus less attractive. More attractive: research on training data synthesis that creates instruction-contrastive pairs (rather than simply more instruction data); work on characterizing and mitigating gradient conflict in multi-objective retrieval training; investigation of what makes instruction pairs "sufficiently contrastive" to force genuine instruction conditioning. Less attractive: the "just scale up" approach to IF-IR — training larger models on larger corpora with instruction data mixed in — since the paper's evidence suggests that scale without signal purity may actively harm IF performance, and even large models (EmbeddingGemma-300M, Qwen3-Embedding-0.6B) fail on IF when trained this way. Also less attractive: architectural solutions to IF (specialized instruction encoders, cross-attention mechanisms, etc.), since the paper shows that a standard bi-encoder with the right data achieves state-of-the-art IF without any architectural modification. The paper effectively argues that data composition, not model design, is the primary bottleneck, which redirects research investment toward data synthesis and mixing strategies.

Follow-Up Research This Work Enables

Characterization of the instruction mixing ratio degradation curve to identify safe mixing regimes. The paper demonstrates catastrophic IF degradation at exactly one non-instruct mixing ratio: 50/50 (All-orig, 440k instruct + 440k non-instruct = 880k total). But practitioners rarely have the luxury of training on instruction-only data — most real-world retrieval training pipelines include substantial non-instruct data from diverse sources, and the paper provides no guidance on what mixing ratios are safe. A direct follow-up experiment would sweep mixing ratios from 0% to 50% non-instruct (e.g., 0%, 5%, 10%, 20%, 35%, 50%) at a fixed total data volume, measuring both IF metrics (FollowIR p-MRR, InfoSearch p-MRR) and general retrieval quality (FollowIR Score) at each ratio. The key questions: Is the degradation linear with non-instruct proportion, or does it exhibit a threshold or phase transition where p-MRR abruptly flips from positive to negative? Does a small amount of non-instruct data (5–10%) provide diversity benefits for general retrieval without measurable IF cost, or is any contamination harmful? If a safe regime exists, practitioners could balance IF and general quality without DV synthesis. If the degradation is catastrophic even at 5–10%, the paper's strong claim that non-instruct data is fundamentally incompatible with IF training would be substantially strengthened.

Mechanistic validation of the gradient conflict hypothesis through direct gradient analysis. The paper explains the data mixing catastrophe through gradient conflict: non-instruct samples push the encoder to represent all topically relevant documents as similar to the query, while instruct samples push it to separate some of those same documents based on instruction constraints, and the former dominates at 50/50 mixing. This mechanism is plausible and consistent with the results, but is never directly tested. A follow-up study would instrument the training process to measure, at each training step, the cosine similarity between the gradient of the InfoNCE loss computed on instruct-only samples versus non-instruct-only samples (computed on held-out validation batches drawn from each distribution). If gradient conflict is the mechanism, these cosine similarities should be consistently negative or near-zero during training (indicating the objectives are pulling in orthogonal or opposing directions), and the magnitude of the conflict should correlate with the degree of IF degradation at different mixing ratios. Additionally, tracking the L2 distance between query-instruction embeddings for paired instructions (I_orig vs. I_new for the same query) over the course of training would directly measure whether the encoder is learning to separate instruction-conditioned representations — the paper's core hypothesized mechanism for why dual-view training works. For All-DV, this distance should increase substantially from initialization; for All-orig, it should remain small (since non-instruct gradients push all query+instruction variants to similar locations). This would convert the paper's inferential claim into an empirically verified mechanism.

Training state-of-the-art IF-IR models by applying dual-view synthesis to large-scale retrieval datasets beyond Promptriever. The paper's experiments are limited to Promptriever's training data as the seed source for both original instruct samples and DV synthesis. But the polarity reversal insight is not Promptriever-specific — any dataset where instruction negatives exist (or can be identified through automated methods) could generate dual-view training pairs. A natural extension would apply the synthesis pipeline to larger, more diverse retrieval training corpora with instruction annotations or instruction negatives that can be automatically detected (e.g., MS MARCO with instruction-conditioned filtering, or the MAIR training data). The key experiment: take a strong retrieval pretrained checkpoint (e.g., a 1B+ parameter encoder), fine-tune it on a large-scale instruct dataset (say, 5M+ samples) with and without DV synthesis at matched volume, and evaluate on the full suite of IF-IR benchmarks. This would test whether the DV benefit scales with data volume and model size — the bge-m3 results in Table 2 showing a larger absolute keyword p-MRR improvement (44.24 points) than the primary backbone (3.55 points) suggest that stronger models may benefit more from DV signal, but this needs systematic verification across model scales. A positive result (consistent IF improvement with DV at scale) would establish DV synthesis as a standard preprocessing step for IF-IR training, similar to how hard negative mining became standard for general retrieval. A negative result (DV benefit saturating or diminishing at scale) would bound the method's applicability and suggest that instruction sensitivity plateaus under current training paradigms.

Stress-testing DV synthesis quality: how sensitive is downstream IF performance to the quality of synthesized complementary instructions? The paper reports 99%+ usability from a 100-sample manual check and applies no filtering, but provides no analysis of what "usable" means, what the <1% failure modes look like, or — critically — whether the model's IF performance degrades gracefully or catastrophically when trained on noisy DV samples. A targeted follow-up would deliberately corrupt a controlled fraction of the DV training data: replace 5%, 10%, 25%, and 50% of the synthesized instructions with random instructions, irrelevant instructions, trivial rewordings of the original instruction (restating the same constraints but phrased differently), or instructions that fail the polarity reversal condition (the original positive remains positive under the new instruction, creating a contradictory positive signal). Training each corrupted variant and measuring IF performance would establish the method's robustness to synthesis quality and identify which type of corruption is most damaging. If trivial rewordings cause similar degradation to random instructions (both producing IF decline), then the polarity reversal property — not just having a second instruction — is what drives DV benefit. If 5–10% corruption causes negligible performance loss, the method is robust enough for production use without manual filtering. If even 1% corruption causes measurable degradation, the 99%+ quality threshold is tight and practitioners may need quality control beyond the paper's current approach.

Testing cross-architecture and cross-task generalization of dual-view training for instruction-following beyond bi-encoders. The paper explicitly limits evaluating on "encoder-based bi-encoder retrievers" and acknowledges that "exploring the applicability to decoder-based or cross-encoder architectures is a natural direction for future work" (Section 6). This follow-up is directly suggested by the paper. For cross-encoders, the experiment is straightforward: take a cross-encoder trained on the same seed data, fine-tune with and without DV synthesis (size-matched), and evaluate on FollowIR p-MRR. Cross-encoders model query-document interaction jointly (through full self-attention between query+instruction and document tokens), which might make them inherently more instruction-sensitive — the instruction can directly attend to and modulate document token representations, potentially reducing the marginal benefit of dual-view training. A null result (DV provides no benefit for cross-encoders) would bound the method's applicability to bi-encoder architectures where query and document representations are computed independently. For decoder-based retrieval (e.g., generative retrieval where a decoder produces document identifiers, or LLM-as-reranker setups), the experiment would adapt DV synthesis to the appropriate training objective (next-token prediction rather than contrastive embedding learning) and test whether presenting complementary instruction-document pairs in the training context improves instruction-following at inference. Additionally, extending to non-retrieval instruction-following tasks (e.g., text generation with constraints, dialogue with personas, summarization with length/style requirements) would test whether polarity reversal is a general principle for teaching models to condition on instructions, or a retrieval-specific phenomenon arising from the contrastive embedding learning dynamic. The key distinction: in retrieval, the "relevance" of a document under an instruction is a binary or rank-based property; in generation, instruction-following involves continuous quality and the training signal is fundamentally different (token-level likelihood vs. contrastive loss). Whether polarity reversal transfers to likelihood-based training is an open question.

Practical Applications and Downstream Use Cases

RAG pipeline quality improvement through instruction-sensitive first-stage retrieval. The paper's results have direct implications for retrieval-augmented generation systems where a chat model formulates queries with specific instruction constraints (e.g., "retrieve documents about climate policy from European sources published after 2020, excluding opinion pieces"). Current RAG pipelines typically use general-purpose embedding models for the retrieval step — and as the paper shows (Table 1), these models achieve p-MRR scores around 5–6 on FollowIR, meaning they are barely better than random at adjusting their rankings when the instruction changes. The downstream consequence is that the retrieved document set is contaminated with instruction-irrelevant documents that match the query topically but not instructionally, causing the generator to produce outputs grounded in wrong-aspect or wrong-format sources. Substituting a DV-trained retriever (FollowIR p-MRR 8.30 for All-DV, Table 1) for a general-purpose one (EmbeddingGemma-300M p-MRR 5.61) provides a 48% relative improvement in instruction sensitivity, which translates directly to more relevant retrieval sets and fewer cascading generation errors. The method is drop-in compatible — since DV synthesis changes only training data, not model architecture or inference pipeline, an organization can train a DV-augmented encoder on their own instruction-annotated retrieval data and swap it into their existing RAG infrastructure without modifying the retrieval index, query processing, or generation components.

Cost-efficient IF-IR training for organizations with limited instruction-annotated data. The paper's finding that dual-view synthesis doubles the number of instruction-conditioned training pairs from a fixed set of instruction-negative-containing instances — with the synthetic pairs providing stronger per-sample IF supervision than original instruct data (Ins-DV outperforms Ins-orig on all IF metrics at the same 480k volume, Table 1) — has direct implications for data-constrained settings. An organization with a modest instruction-annotated retrieval dataset (say, 100k triplets) can use LLM-based polarity reversal synthesis to generate 100k complementary pairs, producing 200k total instruction-conditioned training instances at a one-time LLM inference cost, without additional human annotation. The 45% p-MRR improvement from Ins-DV over Ins-orig at matched 480k volume suggests that each DV sample is substantially more IF-informative than each original instruct sample, so even if the LLM synthesis cost is non-trivial, it may be cheaper per unit of IF improvement than collecting additional human-annotated instruction data. This is particularly relevant for specialized domains (legal, medical, scientific) where instruction annotations require domain expertise and are expensive to obtain — an organization can annotate a small seed set, synthesize complementary instructions via LLM, and train a domain-specific instruction-aware retriever from a modest annotation budget.

Mitigation of data mixing degradation in production retrieval training pipelines. Most production retrieval systems are trained on heterogeneous data mixtures that include large volumes of non-instruction-conditioned query-document pairs (from click logs, web search data, or general-purpose retrieval datasets) alongside smaller volumes of instruction-annotated data. The paper's most practically significant finding is that this mixing — which is standard practice — can catastrophically degrade IF performance (InfoSearch keyword p-MRR crashes from +2.06 to −49.65 when moving from Ins-orig to All-orig, Table 1). The practical recommendation: if a production system must serve both general and instruction-conditioned queries, replace non-instruct supplementary data with DV-synthesized instruction-conditioned pairs rather than mixing in standard retrieval data. The All-DV configuration (440k original + 440k DV, no non-instruct data) achieves better general retrieval quality (Score 21.38) and dramatically better IF (p-MRR 8.30) than All-orig (Score 20.85, p-MRR 5.27) at the same total 880k volume. For an organization with an existing data mixture that includes non-instruct data, the actionable step is to identify which training pairs have associated instruction negatives (or can have them generated), run the polarity reversal LLM synthesis on those pairs, and replace the non-instruct fraction with DV pairs while preserving total training volume. This requires no changes to the training pipeline (same loss, same architecture, same hyperparameters) and avoids the gradient conflict that the paper documents. The improvement is largest on metrics requiring genuine instruction sensitivity (InfoSearch, FollowIR p-MRR), but general retrieval quality is at least maintained and in some configurations improved (Table 1, All-DV Score 21.38 vs. Ins-orig 21.33).

When to Prefer This Method

The paper articulates a specific tradeoff between two data scaling strategies at equal total volume: supplementing instruct data with non-instruct retrieval pairs (All-orig) versus supplementing with dual-view synthetic instruction pairs (All-DV). Based on the results in Tables 1 and 2, the decision rule is:

Prefer dual-view synthesis (All-DV-style training) when: (1) the deployment requires both strong instruction-following sensitivity and good general retrieval quality; (2) you have access to training instances with instruction negatives (or can generate them); (3) you can afford the one-time LLM synthesis cost (unquantified in the paper but offline and non-recurring); and (4) you want to avoid the gradient conflict that standard non-instruct mixing introduces. The evidence: All-DV achieves the best FollowIR p-MRR (8.30, Table 1) and competitive-to-best FollowIR Score (21.38) at the same 880k volume as All-orig, without any non-instruct data or architectural changes.
Prefer instruct-only training without non-instruct mixing (Ins-orig or Ins-DV) when: (1) you are training on a pure instruction dataset with no non-instruct contamination; (2) you prioritize maximizing IF sensitivity over general retrieval quality (Ins-DV), or maximizing general retrieval quality over IF sensitivity (Ins-orig); and (3) training data volume is limited. The evidence: Ins-DV gives the best IF for 480k volume (p-MRR 7.57 vs. 5.21 for Ins-orig) but sacrifices some general quality (Score 19.73 vs. 21.33). Ins-orig gives the reverse tradeoff.
Do NOT mix non-instruct data with instruct data at a 50/50 ratio or similar if IF performance matters: the catastrophic degradation (InfoSearch p-MRR falling to −23.22 and −49.65 in Table 1) means the resulting model will actively contradict instruction-defined relevance on some benchmarks. The paper provides evidence only for this ratio, so mixing at lower ratios is untested in this work — but the severity of the 50/50 degradation warrants caution until further characterization is available.