E2Rank: Your Text Embedding can Also be an Effective and Efficient Listwise Reranker

ArXiv: 2510.22733

🎯 Pitch

A single text embedding model can perform listwise reranking by simply treating the concatenated query and candidate documents as a pseudo-relevance feedback query, matching state-of-the-art LLM rerankers while being up to five times faster—and this same training actually improves retrieval embeddings too.

1. Executive Summary

This paper proposes E2RANK (Efficient Embedding-based Ranking), a unified framework that extends a single text embedding model to perform both retrieval and listwise reranking by training the model to treat the listwise reranking prompt as a pseudo-relevance feedback (PRF) query — enriching the original query with signals from top-K candidate documents so that simple cosine similarity between this enhanced query embedding and precomputed document embeddings serves as the unified ranking function. Evaluated on the BEIR reranking benchmark using Qwen3-family models (0.6B, 4B, 8B), E2RANK achieves state-of-the-art reranking performance — surpassing the directly comparable RankGPT-style baseline RankQwen3 by an average of +4.06 NDCG@10 at 0.6B scale — while delivering up to ~5× inference speedup over autoregressive listwise rerankers, and further demonstrates competitive results on the reasoning-intensive BRIGHT benchmark and maintained embedding quality on MTEB. The approach establishes that a single embedding model can effectively unify retrieval and reranking at low latency, with the boundary that performance gains from listwise PRF signals plateau after roughly 20 input documents and that the framework inherits the base embedding model's capability limitations on the hardest reasoning problems.

2. Context and Motivation

The Core Problem: The Retrieval-Reranking Performance Gap Is Bridged by Computationally Expensive Methods

This paper addresses a fundamental tension in modern information retrieval (IR) systems: embedding-based retrievers are fast but imprecise, while listwise LLM-based rerankers are accurate but computationally prohibitive. The gap between these two stages has become a critical bottleneck as real-world search applications demand both high quality and low latency.

To understand this tension concretely, consider the standard two-stage search architecture that dominates production IR systems (Matveeva et al., 2006). In the first stage, a lightweight embedding retriever maps queries and documents into a shared low-dimensional vector space, enabling approximate nearest neighbor (ANN) search over millions or billions of documents with sub-millisecond latency per query. This is the engine that makes web-scale search feasible. In the second stage, a more powerful reranker takes the top-K candidates (typically 100–1000 documents) and produces a refined ordering that better reflects human relevance judgments.

The problem arises because the two stages use fundamentally different — and unequally powerful — scoring mechanisms. Embedding models represent each query and document as a single fixed-dimensional vector, and relevance is computed as the cosine similarity between these vectors. This is computationally efficient but cannot capture fine-grained interactions between query terms and document content. When a document is relevant only because a specific phrase in paragraph 3 connects to a specific clause in the query, one vector per document simply cannot encode this — the interaction is squeezed through a single dot product.

LLM-based listwise rerankers solve this by placing the entire query and all candidate documents into a single prompt and asking the model to generate a ranking (Sun et al., 2023; Pradeep et al., 2023). The self-attention mechanism in transformer LLMs means every token from every document can attend to every token in the query and every token in competing documents. This enables the model to perform comparative reasoning — "document A is more relevant than document B because it addresses the user's specific constraint while B only discusses the general topic" — which embedding models fundamentally cannot do. The result, as the paper notes, is that listwise methods "can model fine-grained interactions within the entire candidate set and capture both query-document and document-document relationships, leading to rankings that better reflect human judgment."

Why This Problem Matters: The Latency-Cost Barrier to Deployment

The effectiveness gap between retrievers and rerankers would be manageable if rerankers were only marginally more expensive than retrievers. They are not. LLM-based listwise rerankers incur two compounding costs that make them impractical for many real-world deployments:

First, prefilling latency. Encoding a long prompt containing the query plus 20–100 candidate documents (each potentially hundreds of tokens) requires a full forward pass through the LLM. The memory and compute demands scale quadratically with sequence length due to self-attention. For a prompt with 20 documents averaging 350 tokens each plus a query, the total input length can exceed 7,000 tokens — a substantial cost even before any output is generated.

Second, autoregressive decoding latency. RankGPT-style methods require the model to generate a text-form ranking list (e.g., "[2] > [1] > [3]..."). This inherently sequential token-by-token generation cannot be meaningfully parallelized, and the model must produce at minimum 2N tokens to rank N documents (opening brackets, document IDs, closing brackets, ">"). For a candidate set of 100 documents, this is hundreds of output tokens generated one at a time.

The paper quantifies this empirically in Section 4.2: on the TREC COVID dataset using a single NVIDIA A100 80G GPU, RankQwen3-8B requires approximately 16.93 seconds per query for reranking, while E2RANK-8B achieves the same task in approximately 3.40 seconds — and this includes only 2.76 seconds of document encoding that can be done offline and reused. Once document embeddings are precomputed, the online reranking latency drops to just 0.64 seconds per query. The speedup relative to generation-based rerankers reaches approximately 5× at the 8B scale, and remarkably, E2RANK-8B (3.40s) runs faster than RankQwen3-0.6B (4.58s) while delivering substantially higher accuracy.

This latency gap has direct economic and user-experience consequences. In interactive search scenarios, users expect sub-second response times; adding multiple seconds of reranking latency creates perceptible delays that degrade satisfaction. In high-throughput batch processing (e.g., reranking for RAG pipelines, generating training data, or nightly index updates), the cost of running large models over millions of queries becomes prohibitive. The paper is therefore addressing a problem with clear practical urgency: how do we get the ranking quality of listwise LLMs without their computational footprint?

Prior Approaches and Their Limitations

The paper situates itself within three lines of prior work, each of which has made meaningful progress but left a specific gap that E2RANK fills.

LLM-Based Reranking: Effective but Inefficient

The dominant paradigm for high-quality reranking since the emergence of instruction-tuned LLMs has been prompt-based listwise methods, exemplified by RankGPT (Sun et al., 2023) and RankZephyr (Pradeep et al., 2023). These methods construct a prompt containing the query and candidate documents, ask the LLM to output a ranking list, and parse the generated text to obtain the reordered document indices. They achieve state-of-the-art results across multiple benchmarks because the full self-attention over the entire candidate set enables genuine comparative reasoning.

However, as described above, these methods are constrained by the autoregressive generation paradigm. The paper acknowledges a crucial observation from recent work: the autoregressive decoding step is not actually what provides the ranking benefit — it is the interaction between query and documents within the context that matters (Chen et al., 2024b; Zhang et al., 2025b). The model needs to see all candidates together to reason comparatively, but it does not need to generate text about them. Liu et al. (2025b) further showed that incorporating document embeddings into the ranking process is beneficial, suggesting that the embedding space itself carries useful ranking signals.

Several prior works have attempted to improve listwise reranking efficiency, but each addresses only part of the problem:

• Input compression (Liu et al., 2025b) reduces prefilling costs by compressing documents before feeding them to the LLM, but still requires some form of LLM-based scoring.
• Logit or attention-based methods (Reddy et al., 2024; Chen et al., 2024b) extract relevance signals from the LLM's internal representations without autoregressive generation, but still require encoding the full listwise prompt through the LLM.
• Pointwise cross-encoders (monoBERT, monoT5) score each query-document pair independently, achieving efficiency but losing the comparative reasoning that makes listwise methods powerful.

The gap E2RANK identifies is that none of these approaches fully eliminate the need for a heavyweight LLM inference pass at reranking time. The paper's key insight — and the question it explicitly raises — is: "What if incorporating the interaction signals in embedding models for reranking?"

Text Embedding Models: Efficient but Ranking-Limited

Modern text embedding models, particularly those built on LLM backbones (LLM2Vec, E5-Mistral, NV-Embed, Qwen3-Embedding), have achieved impressive retrieval quality through large-scale contrastive learning on curated datasets. They map queries and documents into a shared semantic space where cosine similarity serves as the ranking function, enabling extremely efficient ANN search.

However, these models are trained with objectives — typically InfoNCE loss — that optimize for pointwise query-document alignment. Each document is encoded and scored independently against the query; the model never sees multiple documents together during training and therefore never learns to make comparative distinctions. The consequence is that embedding models plateau in their ranking fidelity: they can determine that a document is topically relevant, but they struggle with fine discrimination among several relevant documents and cannot leverage the mutual information that comes from comparing candidates against each other in context.

The paper does not claim that embedding models are bad per se — they are the backbone of efficient retrieval. But they inherently cannot perform the comparative reasoning that makes listwise reranking so effective, creating the retrieval-reranking gap that the paper seeks to bridge.

Pseudo-Relevance Feedback in Dense Retrieval: A Unexploited Connection

Pseudo-relevance feedback (PRF) is a classic IR technique dating back to Xu and Croft (1996). The idea is simple: after an initial retrieval, assume the top-K retrieved documents are relevant (even though they may not be), extract informative terms or features from these documents, and use them to expand or refine the original query for a second round of retrieval. This creates a form of query enrichment — the query representation bootstraps from the initial retrieval results to become more expressive.

Recent work has adapted PRF to dense retrieval. ANCE-PRF (Yu et al., 2021) feeds the query and top-retrieved documents into a query encoder to produce an improved query embedding, demonstrating that PRF signals can enhance dense representations. However, Li et al. (2022; 2023a) found that this approach is less robust when the base retriever is already strong — the initial retrieval quality is high enough that PRF has limited room to add new information. Other work has applied PRF in rerankers (Li et al., 2024b; Weller et al., 2024), but these approaches use pointwise cross-encoders and require explicit keyword generation for query expansion, adding complexity without the full benefits of listwise comparison.

Critically, no prior work has connected PRF to LLM-based listwise reranking. The paper's conceptual contribution is to recognize that the listwise prompt — the query surrounded by candidate documents — is structurally identical to a PRF-enriched query. When RankGPT constructs "Query: [q], Documents: [d1, d2, ..., dk], Rank the documents," it is effectively performing PRF by providing the model with document context that expands the query's representation. But rather than using this enriched representation to generate a ranking list autoregressively, E2RANK uses it to produce a single embedding — a PRF-enhanced query vector — that can be compared to precomputed document embeddings via cosine similarity. This is the conceptual bridge between the two stages that prior work missed.

How This Paper Positions Itself

The paper positions E2RANK not as an incremental improvement to either retrieval or reranking in isolation, but as a unification that dissolves the traditional boundary between the two stages. Several aspects of this positioning are noteworthy:

It is methodologically different from "improve the reranker" work. Instead of trying to make LLM-based rerankers faster (by compressing inputs, bypassing generation, or distilling to smaller models), the paper asks: can we start from an embedding model — which is already fast by design — and teach it to do listwise reranking? This inverts the typical approach. Rather than removing the slow parts from a powerful reranker, the paper adds powerful ranking capabilities to a fast embedder.

It reinterprets, rather than replaces, existing ideas. The paper does not propose new loss functions or architectures. The RankNet loss dates to 2005 (Burges et al.). PRF dates to 1996 (Xu and Croft). Listwise prompting dates to 2023 (Sun et al., 2023). The contribution is in the reconfiguration: recognizing that the listwise prompt can be treated as a PRF-enriched query, and that cosine similarity in embedding space can serve as a unified ranking function for both retrieval and reranking. This reframing is what enables the two-stage training pipeline — standard contrastive learning followed by multi-task contrastive + ranking training — to produce a single model that does both jobs.

It makes a specific, falsifiable efficiency claim with empirical bounds. The paper does not just claim E2RANK is faster than listwise rerankers — it quantifies the tradeoff (5× speedup at 8B, Figure 1c) and the conditions under which performance scales (gains plateau after ~20 input documents, Figure 2). It also transparently reports where E2RANK underperforms: on the hardest reasoning tasks in BRIGHT, it does not match the strongest RL-trained reasoning rerankers like ReasonRank (trained on synthetic reasoning data), and on certain BEIR datasets (e.g., NFCorpus, DBPedia), it is outperformed by some baselines at specific model sizes (Table 1). This specificity — documenting both strengths and boundaries — distinguishes it from work that reports only aggregate gains.

It connects to a broader vision of unified models. The paper explicitly frames E2RANK as part of a trend toward single models that perform multiple IR tasks, referencing GritLM (Muennighoff et al., 2024) which unified embedding and generation through multi-task learning. The novelty claim is that E2RANK is the first to unify embedding and listwise reranking under a shared embedding space and scoring function, whereas prior unified models maintained separate mechanisms for different tasks.

It recognizes and addresses the "distribution shift" challenge implicitly. The paper notes in Section 6 that training a reranker using the same base model as the embedding model avoids the distribution shift that occurs when using external data (like GPT-4 labels for PRM800k in other work). The two-stage training — contrastive learning on retrieval data followed by ranking-aware training on the same model — keeps the model's representations coherent with its own output distribution, which the paper argues is important for the ranking head (cosine similarity) to remain calibrated.

3. Technical Approach

3.1 Reader Orientation

E2RANK is a system that turns a standard text embedding model into a unified retrieval-and-reranking engine by teaching it to treat a list of candidate documents as pseudo-relevance feedback — extra context that enriches the query representation — so that the exact same cosine similarity computation used for retrieval can also perform high-quality listwise reranking. The problem it solves is the latency-accuracy tension in modern search pipelines: embedding-based retrieval is fast but cannot compare documents against each other, while LLM-based listwise reranking compares documents effectively but requires slow, expensive autoregressive generation; E2RANK resolves this by using the listwise prompt to produce a single enhanced query embedding, eliminating generation entirely while preserving the comparative reasoning that makes listwise reranking powerful.

3.2 Big-Picture Architecture

The system has four major components:

1. Base LLM Backbone (Qwen3 family, decoder-only): A pretrained instruction-tuned language model that serves as the text encoder. It converts any input sequence — a standalone query, a document, or a listwise prompt containing both — into a fixed-dimensional embedding vector by extracting the hidden state at the final [EOS] token position. All downstream scoring is computed as cosine similarity between these embeddings.

2. Two-Stage Training Pipeline: The process that builds the unified model. Stage I trains the backbone as a standard embedding model using contrastive learning on ~1.5M query-document pairs. Stage II continues training with a multi-task objective that combines contrastive loss (to preserve retrieval quality) with RankNet loss (to learn listwise ranking), using the same model weights.

3. Listwise Prompt Constructor: At reranking time, this component assembles the query and top-K candidate documents into a formatted prompt with positional identifiers (e.g., "[1] {doc1}, [2] {doc2}, ...") and task instructions. The resulting prompt is fed into the model to produce a single PRF-enhanced query embedding.

4. Unified Scoring Function: Cosine similarity between the listwise prompt embedding and the precomputed document embeddings. This same function serves both retrieval (query → documents) and reranking (listwise prompt → documents), enabling document embeddings to be computed once and reused across both stages.

Information flows: a search query arrives → Stage I-trained embedding model retrieves top-100 candidates via approximate nearest neighbor search (using query-only embedding) → listwise prompt constructor assembles the query + top-20 documents into a formatted prompt → the Stage II-trained model encodes this prompt into the PRF-enhanced query embedding → cosine similarity between this enhanced embedding and the 100 precomputed document embeddings produces the final reranked ordering.

3.3 Roadmap for the Deep Dive

• First, the embedding model architecture and baseline scoring mechanism (Section 3.1 from the paper), since everything else builds on how the backbone encodes text and computes similarity. This establishes what "cosine similarity between embeddings" means operationally.

• Second, the core conceptual reframing (Section 3.2): how the listwise prompt is reinterpreted as a pseudo-relevance feedback query, why this enables a unified scoring function, and what properties this design inherits.

• Third, Stage I training (Section 3.3, first part): the contrastive learning setup, the InfoNCE loss, the data mixture, and the hyperparameters. This establishes the embedding foundation that Stage II extends.

• Fourth, Stage II training (Section 3.3, second part): the multi-task learning framework, the RankNet loss and why it was chosen over alternatives, the joint objective, the data construction including how ranking labels are generated, and the specific training configurations that make the model learn listwise reranking while preserving retrieval quality.

• Fifth, the inference procedure: exactly what happens when E2RANK performs reranking — the prompt format, the embedding extraction, the scoring, the interaction with first-stage retrieval — and how this differs from both standard embedding retrieval and autoregressive listwise reranking.

3.4 Detailed, Sentence-Based Technical Breakdown

This is primarily a methodology paper whose core idea is that a single embedding model can be trained to perform both retrieval and listwise reranking by treating the listwise prompt as pseudo-relevance feedback, enabling a unified cosine similarity scoring function that eliminates the need for autoregressive generation during reranking.

Embedding Model Architecture and Baseline Scoring

The paper builds on a decoder-only LLM backbone from the Qwen3 family (Yang et al., 2025), used across three sizes: 0.6B, 4B, and 8B parameters. Decoder-only transformers are architecturally simpler than encoder-decoder models (like T5) in that they process input tokens with causal self-attention — each token can only attend to preceding tokens — and produce a sequence of hidden states, one per input position. This causal constraint means that when encoding a document, the model cannot "look ahead" to later tokens, but this limitation is irrelevant for embedding extraction because the relevant representation is taken from the final token position after the full sequence has been processed.

For a given input sequence — whether a query, a document, or a listwise prompt — the model performs a forward pass through all its transformer layers, producing a hidden state vector at every token position. The specific representation used as the sequence embedding is:

$e_d = f(d, \text{[EOS]})[-1]$

where $f$ is the LLM (the embedding model), $d$ is the document text, $\text{[EOS]}$ is the special end-of-sequence token appended to the input, and $[-1]$ denotes taking the hidden state at the final position (corresponding to the [EOS] token) from the last decoder layer.

What it computes: The model reads the entire document token-by-token, updates its internal representation at each step through causal self-attention and feedforward transformations, and the hidden state at the final [EOS] token serves as a compressed summary of the full document — a single fixed-dimensional vector (the model's hidden dimension, which varies by model size: typically 1024 for 0.6B, 2560 for 4B, and 4096 for 8B in the Qwen3 family, though the paper does not explicitly state these dimensions).

Why this form: Extracting the hidden state at the [EOS] position is a widely adopted convention in LLM-based embedding models (LLM2Vec, E5-Mistral, NV-Embed) because the [EOS] token is trained during pretraining to represent the end of a meaningful segment. By appending it at the end of every encoding input, the model learns to accumulate the full sequence information into this position. Alternatives include mean pooling over all token positions or using a dedicated [CLS] token, but the [EOS] approach requires no architectural modifications to the base LLM — it leverages the existing token and training signal.

For queries specifically, the paper follows the instruction-following embedding paradigm introduced by Su et al. (2022): a task-specific instruction string $I$ is prepended to the query before encoding:

$e_q = f(I, q, \text{[EOS]})[-1]$

where $I$ is an instruction like "Given a web search query, retrieve relevant passages that answer the query" and $q$ is the user's search query.

Why this form: Instruction prefixes provide the model with task context that shapes the embedding space. Without the instruction, the same query text might need to serve multiple tasks (retrieval, classification, clustering), but the optimal representation for "find documents about this topic" is different from "what category does this belong to." The instruction acts as a conditioning signal that tells the model how to represent the input. This is important for E2RANK because retrieval and reranking use the same model — the instruction disambiguates which mode the model should be in.

Finally, the relevance score between any query (or listwise prompt) embedding and any document embedding is:

$s(q, d) = \cos(e_q, e_d) = \frac{e_q \cdot e_d}{\|e_q\| \|e_d\|}$

What it computes: The cosine similarity — the dot product of the two vectors after normalizing each to unit length. This is a scalar between -1 (opposite directions) and 1 (identical directions), though in practice, embedding models typically produce positive similarities for relevant pairs (ranging from roughly 0.1 to 0.9). The normalization by vector norms means that only the direction of the embedding matters, not its magnitude, which prevents the model from artificially inflating scores by producing embeddings with large norms.

Why this form: Cosine similarity is the standard in embedding-based retrieval because it enables efficient approximate nearest neighbor (ANN) search algorithms (like FAISS or ScaNN) that can find the most similar documents to a query among millions of candidates in sub-millisecond time. These algorithms rely on the geometric property that cosine similarity can be computed via vector dot products after L2 normalization, enabling indexing structures like product quantization and hierarchical navigable small world graphs. A learned scoring function (e.g., a small MLP on top of embeddings) would be more expressive but would break ANN compatibility — every query would need to be compared against every document, defeating the purpose of embedding-based retrieval.

Core Conceptual Reframing: The Listwise Prompt as Pseudo-Relevance Feedback

The paper's central insight is structural: the prompt used in listwise reranking methods — the query surrounded by candidate documents — can be reinterpreted through the lens of pseudo-relevance feedback (PRF). Rather than treating this prompt as input to a generation model that outputs a ranking list, it can be treated as input to an embedding model that produces an enriched query vector.

To understand this reframing, consider what happens in traditional PRF (Xu and Croft, 1996): after an initial retrieval, the top-K documents are assumed to be relevant (the "pseudo" in pseudo-relevance feedback acknowledges this assumption may be wrong), informative terms or features are extracted from these documents, and these are used to expand the original query — for example, adding the most discriminative terms from the top documents to the query string, or in dense PRF (Yu et al., 2021), feeding the query and top documents together through a query encoder to produce a new query embedding that reflects information from the retrieved set.

The listwise prompt — formatted as "Given a query and some relevant documents, rerank the documents: [1] {doc1} ... [N] {docN} Search Query: {query}" — is structurally a PRF input: the query is enriched by the candidate documents that appear alongside it in context. The self-attention mechanism in the transformer will propagate information between the query tokens and the document tokens, allowing the query representation at the final position to incorporate signals from the documents.

The paper formalizes this for a listwise prompt $\hat{q}$:

$e_{\hat{q}} = f(I, d_1, ..., d_k, q)[-1]$

where $I$ is the instruction (e.g., "Given a web search query and some relevant documents, rerank the documents that answer the query"), $d_1, ..., d_k$ are the candidate documents (typically the top 20 from first-stage retrieval), and $q$ is the original query. The embedding $e_{\hat{q}}$ is extracted from the final [EOS] position exactly as for standard queries.

What it computes: The model reads through the instruction, each document (with positional identifiers like "[1]", "[2]", etc.), and the query, building a representation that contextualizes the query relative to the specific set of candidate documents. Because of causal self-attention, tokens from later documents can attend to tokens from earlier documents and the query, and the query tokens can attend to all preceding document tokens. The final hidden state at the [EOS] token therefore encodes a representation of "this query in the context of these particular documents" — a query enriched by PRF signals.

Why this form: This design preserves three critical properties that alternative approaches sacrifice. First, the scoring function remains cosine similarity — after obtaining $e_{\hat{q}}$, the relevance of each document is simply $\cos(e_{\hat{q}}, e_{d_i})$, the exact same computation used for initial retrieval. This means document embeddings can be precomputed and stored in an ANN index, and the reranking step reduces to computing one query embedding and performing cosine similarity against already-available document embeddings. Second, the listwise prompt provides comparative context — the model sees all candidate documents together, enabling it to implicitly learn relative relevance through the attention mechanism (document A is more relevant than document B because A addresses the specific constraint while B is only topically related). Third, partial candidate sets suffice — the paper shows that feeding only the top-20 documents into the listwise prompt is sufficient to rerank the top-100, because the PRF-enriched query embedding captures distributional information about what "relevant documents look like" for this query, which generalizes to scoring documents not seen in the prompt.

The key contrast is with autoregressive listwise reranking: in RankGPT, the model must generate a text-form ranking ("[2] > [1] > [3]...") which requires token-by-token decoding. In E2RANK, the model produces a single embedding vector, and the ranking is computed by comparing this vector to document embeddings. The comparative information from the listwise prompt is distilled into the query vector itself rather than expressed through generated tokens. This is what the paper means by "the listwise prompt can be transformed into a single PRF-enhanced query embedding, allowing reranking to be efficiently performed via cosine similarity against precomputed document embeddings."

Stage I: Training the Embedding Foundation

The first training stage establishes the model's ability to encode queries and documents into a shared semantic space where relevant pairs have high cosine similarity. This stage is architecturally standard — it follows the contrastive learning paradigm used by E5 (Wang et al., 2023), LLM2Vec (BehnamGhader et al., 2024), and other LLM-based embedding models — but it provides the foundation that Stage II extends with ranking capabilities.

Training data. The paper uses the public portion of the E5 training dataset (Wang et al., 2023), specifically a sampled version with approximately 1.5 million query-document pairs curated by Springer et al. (2025) and used by LLM2Vec. The data mixture spans diverse retrieval tasks: ELI5 (explanation-seeking questions), HotpotQA (multi-hop reasoning), FEVER (fact verification), MIRACL (multilingual retrieval), MS MARCO passage and document ranking, Natural Questions, SQuAD (reading comprehension), TriviaQA, Quora duplicate questions, Mr.TyDi (multilingual), DuReader (Chinese retrieval), and T2Ranking (Chinese passage ranking). Each instance contains one query, one positive document (known to be relevant), and one negative document (known to be irrelevant or randomly sampled).

Loss function. The model is trained with the standard InfoNCE loss (Izacard et al., 2021), which is a contrastive objective that pulls positive query-document pairs together in embedding space while pushing negative pairs apart:

$\mathcal{L}_{\text{InfoNCE}} = -\frac{1}{N} \sum_{i=1}^{N} \log \frac{e^{s(q_i, d_i^+)/\tau}}{e^{s(q_i, d_i^+)/\tau} + \sum_{d_j \in D^-} e^{s(q_i, d_j)/\tau}}$

where $N$ is the number of queries in the batch, $s(q_i, d_i^+)$ is the cosine similarity between query $q_i$ and its positive document $d_i^+$, $D^-$ is the set of negative documents (which includes all other documents in the batch used as in-batch negatives, plus any hard negatives provided in the training data), and $\tau$ is a temperature hyperparameter controlling the sharpness of the softmax distribution, set to $\tau = 0.03$ during Stage I training.

What it computes: For each query in the batch, the model computes the cosine similarity to its positive document (the one known to be relevant) and to all negative documents (both explicit negatives from the data and implicit negatives from other queries' documents in the same batch). These similarities are divided by the temperature and exponentiated, producing positive values. The fraction inside the log is the ratio of the positive pair's exponentiated similarity to the sum of all exponentiated similarities — this is the probability that the model assigns to the positive document being the relevant one among all candidates. The loss is the negative log of this probability, averaged over the batch. Minimizing this loss encourages the model to maximize the similarity for positive pairs (making the numerator large) while minimizing similarity for negative pairs (making the denominator terms small), effectively performing an N-way classification where each query must correctly identify its one positive document among all candidates.

Why this form: InfoNCE is the de facto standard for contrastive representation learning because it has strong theoretical connections to mutual information maximization — minimizing the InfoNCE loss is equivalent to maximizing a lower bound on the mutual information between the query and document representations. The temperature $\tau$ controls the concentration of the distribution: a small $\tau$ (0.03 is relatively small) makes the model focus heavily on the hardest negatives (those with similarity close to the positive) because small temperature differences get amplified in the exponent. This leads to better separation between relevant and near-relevant documents. Alternatives like triplet loss would only compare the positive to a single negative at a time, losing the global comparison structure that makes contrastive learning effective with large negative sets.

Why in-batch negatives are used: In a batch of size $B$, each query has access to $B-1$ negative documents — the positives for all other queries in the batch. This provides a large number of diverse negatives at no additional computational cost (the documents are already encoded for their own query's positive pair). The effective negative set size scales with batch size, which is a key reason the paper uses a batch size of 512 — this provides 511 negative comparisons per query, substantially more than the 1 explicit negative in the data.

Training configuration. The Stage I training uses full parameter fine-tuning (all weights updated, not just a subset) with the following hyperparameters:

• Batch size: 512 (effective)
• Training duration: 1 epoch over the ~1.5M samples
• Learning rate: $2 \times 10^{-5}$
• Learning rate schedule: Linear warmup for the first 300 steps, then linear decay
• Maximum sequence length: 512 tokens
• Hardware: 8 NVIDIA A100 80G GPUs
• Optimization: DeepSpeed ZeRO-3 (parameter sharding across GPUs), BF16 mixed precision, gradient checkpointing (trading compute for memory by recomputing activations during the backward pass rather than storing them)

The use of DeepSpeed ZeRO-3 is notable: it partitions not just optimizer states and gradients (as in ZeRO-1/2) but also model parameters across GPUs, enabling full-parameter fine-tuning of an 8B model on 8 GPUs. BF16 mixed precision reduces memory usage and accelerates computation while maintaining adequate numerical precision for training stability — the brain floating point format has the same exponent range as FP32 but only 7 bits of mantissa (versus 23 for FP32), which is sufficient for most deep learning operations. Gradient checkpointing is necessary because the 512-token sequence length with a large model would otherwise require storing activations for all transformer layers, which would exceed GPU memory.

What Stage I produces: A model that can perform embedding-based retrieval — given a query (with instruction prefix), encode it to a vector, and use cosine similarity to rank documents from an ANN index. The paper reports Stage I-only performance on MTEB as a baseline: 62.40 average for 0.6B, 65.33 for 4B, and 65.96 for 8B (Table 4, "w/ only Stage I" rows). These numbers are competitive with contemporary embedding models, confirming that Stage I establishes a solid retrieval foundation.

Stage II: Multi-Task Training for Unified Retrieval and Listwise Reranking

The second training stage is where E2RANK acquires its distinctive capability: performing listwise reranking through the embedding mechanism. This stage continues training from the Stage I checkpoint with a multi-task objective that jointly optimizes contrastive learning (to preserve retrieval quality) and a ranking-specific loss (to learn comparative document ordering).

Training data. Stage II requires richer training instances than Stage I because the RankNet loss needs multiple documents with known relative relevance orders. The paper constructs a new dataset by intersecting the E5 training datasets with the BGE-M3 training dataset (Chen et al., 2024a), which provides queries with multiple negatives (unlike the single-negative E5 data). The intersecting datasets include HotpotQA, MIRACL, MSMARCO passage, NQ, TriviaQA, DuReader, and T2Ranking. Additionally, two Chinese retrieval datasets from BGE-M3 are added: cMedQAv2 (medical question answering) and MMarco Chinese (Chinese translation of MS MARCO). Documents with length exceeding 500 tokens are filtered out, queries with fewer than 15 negatives are removed, and the resulting dataset is downsampled. From this filtered set, at most 10,000 instances are sampled per dataset, yielding approximately 87,000 training instances. Each instance contains 1 query, 1 positive document, and 15 negatives.

Why this data construction: The multiple negatives are essential for the RankNet loss — with only 1 positive and 1 negative, the model would only learn a single pairwise comparison per query, providing limited ranking signal. Fifteen negatives per query enables the model to learn a richer relative ordering across many document pairs. The filtering to documents under 500 tokens reflects the practical constraint that listwise prompts in Stage II training pack multiple documents into one sequence (the listwise prompt), and longer documents would exceed the maximum sequence length or force truncation that loses important content.

Generating ranking labels. To train a listwise reranker, the model needs to know what the correct ranking order is — not just which documents are relevant versus irrelevant, but the full relative ordering among all 16 documents (1 positive + 15 negatives) for each query. The paper generates these labels by leveraging a much larger LLM, Qwen3-32B with thinking mode disabled, following the procedure from RankZephyr (Pradeep et al., 2023).

The labeling process works as follows: for each training instance, the query and all 16 documents are formatted into a listwise prompt (identical in structure to the inference prompt) and fed to Qwen3-32B. The prompt includes an instruction to "Rank the passages based on their relevance to the search query" with an output format specification: [] > [] > .... The model generates a text-form permutation of document identifiers, which is parsed to extract the ranked order. Instances where the LLM produces incorrectly formatted output (a small fraction) are filtered out. This produces a complete ground-truth ranking that serves as the supervision signal for the RankNet loss.

An interesting empirical finding the paper notes in Appendix B: the LLM's top-ranked document matches the dataset's "golden positive" document with varying consistency across datasets. For example, MS MARCO shows only 54.3% agreement between the LLM label and the original dataset positive, while HotpotQA shows 91.3% agreement. This discrepancy likely arises because the dataset positives were created under different annotation guidelines than what the LLM considers most relevant, but the paper defers analysis of this phenomenon to future work. For training purposes, the LLM-generated ranking is treated as ground truth, following the standard approach in RankZephyr-style training.

The RankNet loss. The core ranking objective in Stage II is the RankNet loss (Burges et al., 2005), a pairwise learning-to-rank loss that penalizes incorrectly ordered document pairs:

$\mathcal{L}_{\text{RankNet}} = \frac{1}{N} \sum_{i=1}^{N} \sum_{d_j \in D} \sum_{\substack{d_k \in D \\ r_j < r_k}} \log\left(1 + e^{s(q_i, d_j)/\tau - s(q_i, d_k)/\tau}\right)$

where $N$ is the number of queries in the batch, $D$ is the set of documents for query $q_i$ (including both positive and all negatives — typically 16 documents), $s(q_i, d_j)$ is the cosine similarity between the query embedding and document $d_j$'s embedding, $r_j$ is the rank of document $d_j$ in the ground-truth ordering (with 1 being most relevant and higher numbers being less relevant), $\tau$ is a temperature parameter set to 0.1, and the inner summation is over all pairs $(d_j, d_k)$ where $d_j$ is ranked higher than $d_k$ in the ground truth (i.e., $r_j < r_k$).

What it computes: For every pair of documents where the ground truth says document $j$ should be ranked above document $k$ (i.e., $j$ is more relevant than $k$), the model computes the difference in their similarities: $s(q_i, d_j)/\tau - s(q_i, d_k)/\tau$. The exponentiated negative of this difference, $\exp(s(q_i, d_j)/\tau - s(q_i, d_k)/\tau)$, feeds into a log-sigmoid loss. If $s(q_i, d_j) \gg s(q_i, d_k)$ (the model strongly prefers the correct ordering), $\exp(\text{positive large number})$ is large, but $\log(1 + \text{large})$ ≈ the large number — a high loss because the model is overconfident? No: let's trace this more carefully:

The term is $\log(1 + \exp(\Delta))$ where $\Delta = s(d_j)/\tau - s(d_k)/\tau$. If $\Delta$ is large and positive (model assigns much higher similarity to the correctly higher-ranked document), $\exp(\Delta)$ is large and $\log(1 + \exp(\Delta)) \approx \Delta$, which is large — but this seems wrong because the model is doing the right thing. The key insight is the standard RankNet formulation rearranges this to penalize the violation: the loss should be low when the higher-ranked document has higher similarity. Looking more carefully at the paper's equation and the original RankNet paper, the standard form includes the sigmoid of the negative difference: $\log(1 + \exp(-(\sigma(s_j) - \sigma(s_k))))$ where $\sigma$ is the model's predicted score. The paper's formulation $\log(1 + \exp(s(d_j)/\tau - s(d_k)/\tau))$ with the condition $r_j < r_k$ (meaning $d_j$ should rank above $d_k$) would produce a large loss when $s(d_j) > s(d_k)$, which is the opposite of what we want. I need to examine this more carefully.

Re-reading the condition: $r_j < r_k$ where "the smaller the rank, the more relevant." So $d_j$ with rank 2 is more relevant than $d_k$ with rank 5. The term $\log(1 + \exp(s(d_j)/\tau - s(d_k)/\tau))$ has a large value when $s(d_j) \gg s(d_k)$ — exactly the situation where the model is correct. But that would mean the loss is maximized when the model is correct, which can't be right.

The resolution is that this is likely an editorial choice in how the condition's inequality is written. In standard RankNet, the loss is computed on $\log(1 + \exp(-(s_j - s_k)))$ when document $j$ should rank above document $k$. If the paper instead accumulated $\log(1 + \exp(s_j - s_k))$ for $r_j > r_k$ conditions, the math would work equivalently. Since the paper reports the loss works effectively and the qualitative behavior of RankNet is well-understood, I interpret the loss as: for any pair where the ground-truth ranking is violated (a lower-ranked document gets a higher similarity score), the model incurs a penalty proportional to $\log(1 + \exp(|\text{difference}|))$. When the predicted ordering matches the ground truth, the loss is near zero.

For training, the loss is computed over all $\binom{|D|}{2}$ valid pairs in the document set $D$ (roughly 120 pairs for 16 documents), then averaged across all queries in the batch.

Why this form: RankNet is chosen over several alternatives for specific reasons. First, it is a pairwise loss — it compares documents two at a time and penalizes ordering violations — which is more fine-grained than listwise losses that operate on entire probability distributions over permutations. Second, unlike LambdaRank (which weights pairs by the change in NDCG from swapping them), RankNet treats all violations equally, which is simpler and has been shown to work well in prior IR work. Third, RankNet operates directly on the model's similarity scores without requiring the scores to be calibrated probabilities — the sigmoid of the score difference acts as an implicit probability that one document is more relevant than another. Fourth, compared to pointwise losses (which treat each query-document pair independently), RankNet enables the model to learn relative ordering, which is exactly what reranking requires. The temperature $\tau = 0.1$ (compared to 0.03 in Contrastive loss) scales the similarity differences — a smaller temperature makes the sigmoid steeper, meaning the model only needs small differences in similarity to produce confident pairwise predictions.

The joint training objective. Stage II combines both losses with a weighting hyperparameter $\lambda$:

$\mathcal{L} = \mathcal{L}_{\text{InfoNCE}} + \lambda \mathcal{L}_{\text{RankNet}}$

where $\lambda = 2.0$, determined by prior experiments (the paper does not provide a sweep or justification for this specific value).

What it computes: The model simultaneously optimizes two objectives on the same batch: the standard contrastive loss that aligns queries with positive documents relative to negatives (preserving the embedding model's retrieval capability), and the RankNet loss that ensures the model's similarity scores respect the pairwise ordering from the LLM-generated ground truth (learning the listwise reranking capability). The weighting $\lambda = 2.0$ means the ranking loss contributes twice as much to the gradient as the contrastive loss, reflecting the paper's prioritization of learning reranking while not abandoning retrieval.

Why this multi-task design: The ablation study (Table 6) demonstrates that removing either component degrades performance. Removing InfoNCE in Stage II ("w/o InfoNCE in Stage II") causes a modest drop on BEIR (52.09 → 52.17? actually, reading the table, w/o InfoNCE gets 52.17 vs. full 52.09 on BEIR — a slight improvement — but causes drops on BRIGHT: 30.96 → 29.99) and a more substantial drop on MTEB (63.41 → 61.92), showing that the contrastive loss is essential for maintaining embedding quality. Conversely, removing RankNet ("w/o RankNet in Stage II") causes a severe collapse on BEIR (52.09 → 49.24) and BRIGHT (30.96 → 22.40), demonstrating that the ranking loss is indispensable for the reranking capability. The multi-task design enables the model to serve both roles with a single set of weights.

On a more technical note: the RankNet loss requires the model to score each document against the listwise prompt embedding — but the listwise prompt contains all documents. This creates a potential information leak: the model knows about document $d_j$ because it appeared in the prompt, so scoring $d_j$ against the prompt embedding could be trivially high (the model could just "recognize" it). The paper does not address this directly, but it's inherent to the PRF mechanism: the enriched query embedding encodes document information, and the model must learn to use this to produce discriminative scores rather than simply boosting all documents that appeared in the prompt equally. The fact that the model achieves discriminative performance (differentiating between top-20 prompt documents and top-100 unseen documents, and producing different scores for different prompt documents) confirms that it learns meaningful comparative scoring rather than a trivial "was in prompt" signal.

Stage II training configuration. The training uses the following hyperparameters:

• Batch size: 128 (effective), where each instance in the batch contains 1 query and its full document set
• Training duration: ~700 steps (approximately 55,000 queries processed, given 87k total instances and 128 batch size)
• Learning rate: $5 \times 10^{-6}$
• Learning rate schedule: Linear warmup with warmup ratio 0.03, then linear decay
• Maximum document length: 1024 tokens (for documents in the listwise prompt)
• Negatives per query: 15 (plus in-batch negatives from other queries)
• Hardware: 8 NVIDIA A100 80G GPUs
• Optimization: DeepSpeed ZeRO-3, BF16 mixed precision, gradient checkpointing

The learning rate reduction from $2 \times 10^{-5}$ in Stage I to $5 \times 10^{-6}$ in Stage II is typical for continued training — a smaller learning rate prevents catastrophic forgetting of Stage I representations while still allowing the model to adapt to the new ranking objective. The maximum document length increase from 512 to 1024 reflects the need to encode full documents (not truncated) in the listwise prompts. The batch size of 128 with 16 documents each means the model processes 2,048 documents per batch (128 × 16), which is comparable in document volume to the Stage I batch size of 512 (512 × 1 document each).

The instruction used in Stage II training. The listwise prompts in Stage II use dataset-specific instructions (Table 9 in Appendix C), for example: "Given a web search query and some relevant documents, rerank the documents that answer the query" for MSMARCO, or "Given a multi-hop question and some relevant documents, rerank the documents that answer the question" for HotpotQA. These instructions are distinct from the query-only instructions used for retrieval, which signals to the model that the listwise prompt is a different task that should be processed differently in the embedding space. The use of diverse instructions across datasets may help the model learn a generalizable "reranking mode" rather than overfitting to a single instruction format.

Inference Procedure: How E2RANK Performs Reranking

At inference time, E2RANK operates in two distinct modes — retrieval mode and reranking mode — using the same model weights but different input formatting.

Retrieval mode (first stage). This is identical to a standard embedding model. The query is wrapped with its task-specific instruction (e.g., "Given a web search query, retrieve relevant passages that answer the query") and the [EOS] token, then encoded to produce $e_q$. Document embeddings are precomputed (encoded once) and indexed. Approximate nearest neighbor search returns the top-K candidates (typically K = 100 in the paper's experiments). This stage uses the model's Stage I-contrastive-learned representations.

Reranking mode (second stage). This is where E2RANK's novel contribution manifests. The top-M documents from the first stage (M = 20 by default, chosen based on the analysis in Figure 2 showing performance plateaus after 20 documents) are assembled into the listwise prompt:

<|im_start|>user
Given a web search query and some relevant documents,
rerank the documents that answer the query:
Documents:
[1] {document 1 text}
[2] {document 2 text}
...
[20] {document 20 text}
Search Query:
{query}
<|im_end|>
<|im_start|>assistant
 thinking\n\n response\n\n

The paper notes that the chat template is applied for listwise prompts, which is a distinction from standard embedding encoding where only the instruction + input format is used without the full chat structure. The \<im_start\>user, \<im_end\>, and \<im_start\>assistant tokens are from the Qwen3 chat format and signal to the model that this is a conversational instruction-following context, which may influence how the model represents the sequence.

The model encodes this entire prompt and extracts the hidden state at the final [EOS] position (which appears after "response\n\n" in the assistant turn) to obtain the PRF-enhanced query embedding $e_{\hat{q}}$. Then, for each of the top-K documents (K = 100), the reranking score is:

$s(\hat{q}, d_i) = \cos(e_{\hat{q}}, e_{d_i})$

and documents are sorted in descending order of this score. The document embeddings $e_{d_i}$ are the same embeddings used in the retrieval stage — they are computed once and reused for both retrieval and reranking, which is a key efficiency advantage.

What happens computationally: The model encodes the listwise prompt (one forward pass through the LLM) and computes K cosine similarities (each is a dot product of two vectors, computationally negligible compared to the forward pass). The total cost is dominated by encoding the listwise prompt, whose length depends on the number and length of included documents. For 20 documents averaging ~350 tokens each, plus formatting, the prompt length is roughly 7,000–8,000 tokens — comparable to encoding a single long document. This is substantially cheaper than autoregressive generation (which would require both encoding the prompt and generating hundreds of output tokens), and substantially cheaper than running a separate forward pass for each query-document pair (as in pointwise cross-encoders, which would need 100 forward passes to rerank 100 documents).

Why partial candidate feeding works. One of the paper's key design choices is feeding only top-20 documents into the listwise prompt to rerank the top-100. This works because the PRF mechanism enriches the query representation with information about what "relevant documents look like" for this specific query, using the top-20 as exemplars. The enriched query embedding $e_{\hat{q}}$ encodes not just the query semantics but also the distributional properties of documents that the initial retriever deemed promising. When this enriched embedding is compared against document 57 (which was not in the prompt), it can leverage the comparative signals learned during training — the model has learned to represent queries in a way that generalizes document comparison beyond the specific documents seen in the prompt. The analysis in Figure 2 confirms this empirically: performance with 100 input documents (all candidates in the prompt) is nearly identical to performance with 20 input documents on DL19 and DL20, demonstrating that the PRF signal from 20 documents sufficiently characterizes the relevance space.

Why not feed all 100 documents? Beyond the plateau in effectiveness, there are two practical reasons. First, encoding 100 documents in a single prompt would produce a sequence length of roughly 35,000–40,000 tokens, which approaches or exceeds the context window limits of many models and incurs quadratic self-attention costs that would erode the efficiency gains. Second, the paper's analysis (Figure 2) shows that additional documents beyond ~20 can sometimes degrade performance on certain datasets (the curve is not monotonically increasing), possibly because including lower-quality documents from ranks 20–100 introduces noisy PRF signals that distort the query representation.

Answer selection mechanism. Unlike RankGPT, which generates a permutation string that must be parsed, E2RANK produces a real-valued score for each document. This has two practical advantages beyond efficiency: (1) there is no parsing ambiguity — scores are directly comparable, and (2) documents with identical or near-identical scores (ties) can be handled naturally through score sorting rather than requiring the model to output tied ranks. The paper does not explicitly discuss tie-breaking, but the continuous scoring mechanism makes ties vanishingly unlikely at floating-point precision.

Summary of Design Choices and Their Justifications

• Decoder-only LLM backbone with [EOS] token embedding extraction: Leverages existing pretrained models without architectural modification; the [EOS] position naturally accumulates sequence-level information during causal self-attention; enables the model to serve as both retriever and reranker with the same architecture and weights.

• Two-stage training (contrastive then multi-task): Prevents the ranking objective from interfering with the initial formation of a good embedding space; contrastive learning benefits from large-scale weakly supervised data (1.5M pairs), while ranking training requires richer per-query labels (16 documents with full ranking) that are more expensive to obtain at scale; enables Stage I to be trained once and Stage II to be iterated or adapted independently.

• RankNet loss rather than listwise or pointwise losses: Pairwise losses are simpler and more stable than full listwise losses (which must model the entire permutation space); RankNet specifically operates on the sigmoid of similarity differences, providing a probabilistic interpretation of relative relevance; compared to LambdaRank, RankNet avoids the complexity of computing NDCG gradients while still capturing the essential pairwise ranking signal.

• Temperature 0.1 in RankNet vs. 0.03 in InfoNCE: RankNet's temperature scales similarity differences between documents — at 0.1, even moderate differences in similarity produce confident pairwise predictions, which is appropriate because the ranking task requires fine discrimination among many documents. The InfoNCE temperature of 0.03 is more aggressive, heavily penalizing hard negatives, which is beneficial for retrieval where the model must distinguish the one relevant document from millions of candidates.

• Multi-task weighting $\lambda = 2.0$: Prioritizes ranking capability while maintaining retrieval quality; the 2:1 ratio reflects the paper's goal of producing a model that excels at both but with emphasis on the novel reranking contribution. The ablation shows this balance is effective but does not explore whether other ratios would be better.

• LLM-generated ranking labels rather than dataset ground truth: The dataset's original labels typically provide only binary relevance (relevant/irrelevant) or a small number of graded levels, insufficient to produce a full permutation ranking; using a powerful LLM to generate complete rankings provides richer training signal consistent with the listwise reranking objective. This choice does introduce label noise (as shown by the 54.3% MS MARCO agreement rate), but the model's strong empirical performance suggests the ranking signal is sufficient despite the noise.

• Chat templates for listwise prompts but not for standard embedding: The chat template signals instruction-following mode, which may help the model activate its conversational reasoning capabilities (trained during the base LLM's instruction tuning) when processing the complex listwise prompt. For standard embedding, the simpler format reduces token overhead and aligns with how embedding models are typically used.

• Top-20 documents in the listwise prompt to rerank top-100: Balances the information gain from PRF (more documents provide richer signals) against computational cost (sequence length scales with document count) and the risk of noisy feedback (lower-ranked documents are less likely to be relevant, introducing misleading PRF signals). The empirical plateau at 20 documents (Figure 2) validates this as a sweet spot on the evaluated datasets.

• Document embedding reuse across retrieval and reranking: A single forward pass per document serves both stages, eliminating the redundant computation that would occur if the reranker encoded documents separately. This is structurally impossible for autoregressive listwise rerankers, which must encode documents in the prompt context each time.

4. Key Insights and Innovations

Innovation 1: Reframing Listwise Reranking as Embedding-Based Pseudo-Relevance Feedback

The paper's most conceptually distinctive move is not a new architecture or loss function — it is a reframing that dissolves the boundary between retrieval and reranking by recognizing a structural identity that prior work missed. Before E2RANK, the field treated listwise reranking as a generation problem: you give an LLM a prompt containing the query and candidate documents, and it generates a ranking list (Sun et al., 2023; Pradeep et al., 2023). Subsequent efficiency improvements operated within this frame by compressing the prompt, bypassing generation, or extracting ranking signals from internal LLM states (Liu et al., ; Reddy et al., 2024; Chen et al., ). But all these approaches preserved the fundamental assumption that listwise reranking requires some form of LLM inference over the full candidate set — whether generation-based or representation-based — separate from the embedding retriever.

E2RANK breaks this assumption by recognizing that the listwise prompt — the query surrounded by candidate documents — is structurally identical to a pseudo-relevance feedback (PRF) query. In classical IR, PRF enriches a query with terms extracted from top-retrieved documents (Xu and Croft, 1996). In dense PRF, the query and top documents are jointly encoded to produce an improved query vector (Yu et al., 2021). The listwise prompt does exactly this: the query is contextualized by documents that appear alongside it in the input, and the self-attention mechanism propagates document information into the query representation. The conceptual reframing is: "the listwise prompt is a PRF-enriched query, and the enriched query embedding can score documents via cosine similarity — the exact same mechanism used for retrieval."

This insight has three cascading consequences that separate it from prior reframings:

First, it enables a truly unified scoring function. Prior work on unified models, such as GritLM (Muennighoff et al., 2024), unified embedding and generative abilities through multi-task training, but maintained separate mechanisms for each task — embeddings for retrieval, autoregressive decoding for generation. E2RANK unifies retrieval and reranking under the exact same operation: cosine similarity between query embedding and document embeddings. The only difference is that the query embedding is computed from a query-only prompt for retrieval and from a listwise prompt for reranking. This is a genuinely unified model — not a model that can do two things, but a model that does two things through the same mechanism.

Second, it eliminates the need for any LLM inference at reranking time beyond a single embedding extraction. This is not an incremental efficiency improvement — it is a categorical shift in the computational cost profile. Standard listwise reranking requires either autoregressive generation (RankGPT) or full-prompt encoding with some form of output extraction (logit-based, attention-based methods). E2RANK requires exactly one forward pass to produce the enriched query embedding, plus K dot products against precomputed document embeddings. The paper quantifies the impact: approximately 5× speedup over generation-based reranking at 8B scale, with E2RANK-8B running faster than RankQwen3-0.6B while delivering substantially higher accuracy (Figure 1c, Table 1).

Third, it provides a principled explanation for why partial candidate feeding works. The paper's empirical finding that feeding only top-20 documents into the listwise prompt suffices to rerank top-100 (Figure 2) is not just a practical optimization — it follows directly from the PRF interpretation. PRF theory says that the top-retrieved documents provide distributional information about what relevant documents look like for this query. The enriched query embedding captures the category of relevant content, not just a ranking of the specific documents in the prompt. This means the enriched embedding generalizes to score documents not seen in the prompt — exactly what the PRF framework predicts and what the experiments confirm.

The significance of this reframing extends beyond the specific method. It suggests that the field's default mental model — retrieval and reranking are separate stages requiring separate mechanisms — may be an artifact of how the technologies developed (embeddings for speed, LLMs for quality) rather than a fundamental architectural requirement. If PRF signals can be encoded into an embedding, then the embedding space itself can serve as the universal medium for relevance scoring, with the quality of scoring determined by how enriched the query representation is, not by which family of model produces it.

Innovation 2: The Diagnosis That Generative Decoding Is Incidental, Not Essential, to Listwise Reranking

Prior to E2RANK, the dominant family of state-of-the-art listwise rerankers — RankGPT, RankZephyr, RankVicuna — all used autoregressive generation to produce ranking lists. This design choice created a natural but ultimately misleading assumption: that the generation process was somehow integral to the ranking quality. After all, these models achieved the best results, and they all generated text. Subsequent efficiency work operated within this assumption, trying to make generation faster (e.g., by constraining the output vocabulary) or by extracting the relevant signal while avoiding generation (Reddy et al., 2024; Chen et al., ).

E2RANK makes a sharper diagnostic move: it explicitly identifies, through reference to prior observations, that generation is incidental — the real source of ranking power is the interaction between query and documents within the shared context. The paper cites Chen et al. () and Zhang et al. () as establishing that "the auto-regressive generation paradigm adopted by RankGPT is not necessary for ranking, while the interaction between query and documents in the context is critical for ranking effectiveness." E2RANK then operationalizes this diagnosis in the most extreme way possible: it removes generation entirely and replaces it with embedding extraction, showing that the ranking quality is not only preserved but (at matched model sizes) improved.

This is more than an efficiency improvement — it is a diagnostic result about what makes listwise reranking work. The mechanism is context-based comparative reasoning, not token-by-token deliberation. The self-attention over the joint query-document input provides the model with the ability to compare documents against each other and against the query simultaneously. The generation step in RankGPT merely externalizes a ranking that was already implicitly determined by the model's internal representations after encoding the prompt. E2RANK shows that this internal representation — specifically, the final hidden state at the [EOS] token — can be projected directly into a scoring function without loss of ranking fidelity.

The evidence for this diagnosis is not just the aggregate performance numbers but the difficulty-dependent patterns. On the BEIR benchmark, E2RANK-0.6B outperforms RankQwen3-0.6B by +4.06 NDCG@10 on average (Table 1), demonstrating that the embedding-based approach can be more effective than generation-based reranking at the same model size. The gains are most pronounced on datasets requiring fine discrimination among topically similar documents — for example, +8.32 on TREC News and +7.36 on Robust04 for the 0.6B model — suggesting that the RankNet-trained embedding space captures comparative relevance signals more effectively than whatever ranking information survives the generation bottleneck.

This innovation reframes the research agenda for reranking: rather than asking "how can we make generation faster?", the question becomes "how can we better encode comparative document information into the query representation?" — a fundamentally different and potentially more tractable problem.

Innovation 3: A Training Recipe That Inverts the Typical Reranker Development Pipeline

The standard approach to building listwise rerankers has been: take a powerful instruction-tuned LLM, prompt it to generate rankings, and optionally fine-tune it on ranking data. This treats ranking as a downstream application of general-purpose LLM capabilities — the model's reasoning and instruction-following abilities are leveraged to produce rankings as a special case of text generation.

E2RANK inverts this pipeline: it starts from an embedding model — which is already optimized for efficient relevance scoring — and teaches it to incorporate the comparative interaction signals that make listwise reranking powerful. This inversion is not just a different starting point; it reflects a different philosophy about what ranking capability fundamentally requires. The embedding model already knows how to represent documents and queries in a shared space where similarity correlates with relevance. What it lacks is the ability to use information from multiple documents to refine its judgment. The two-stage training process — contrastive learning followed by multi-task contrastive + ranking training — is designed to add this capability without destroying the embedding foundation.

This approach has three practical implications that distinguish it from fine-tuning-based reranker development:

First, it uses substantially less ranking-specific training data. RankZephyr-style training typically requires tens of thousands of LLM-labeled ranking examples to teach a general-purpose LLM to perform listwise reranking. E2RANK's Stage II uses approximately 87,000 examples — comparable in volume — but these examples are used only to teach the embedding model to incorporate comparative signals, not to learn the entire concept of relevance from scratch. The Stage I model already understands relevance; Stage II teaches it to refine that understanding using document context. The ablation in Table 6 confirms this: removing Stage I and training only with the ranking objective causes modest drops on reranking (51.33 vs. 52.09 on BEIR) but catastrophic drops on embedding quality (60.61 vs. 63.41 on MTEB v2), demonstrating that the embedding foundation is load-bearing for the unified model.

Second, it avoids the distribution shift problem that plagues fine-tuned rerankers. When a general-purpose LLM is fine-tuned solely on ranking data, its representations adapt to the specific characteristics of that data — the document lengths, the relevance patterns, the instruction formats. This can cause the fine-tuned model to underperform on out-of-distribution ranking tasks or to lose capabilities it had from pretraining. E2RANK's multi-task objective (InfoNCE + RankNet) constrains the model to remain in a region of weight space where retrieval quality is preserved. The MTEB results (Table 4) show that Stage II training not only maintains embedding performance but actually improves retrieval: E2RANK-8B achieves 56.89 retrieval NDCG versus 55.31 for the Stage I-only variant, a +1.58 gain. This means the ranking training provides a form of positive transfer — the comparative signals that help reranking also refine the embedding space for retrieval.

Third, it produces a single model that can serve both stages of a search pipeline. This is not just an engineering convenience; it reduces the system complexity, memory footprint, and maintenance burden of production search systems. Instead of deploying separate retriever and reranker models (often using different architectures, frameworks, and serving infrastructure), a single E2RANK model handles both stages. The end-to-end results in Table 5 demonstrate that this unified approach is viable: using the same model for retrieval and reranking produces consistent gains over retrieval alone across all model sizes and benchmarks.

What makes this an innovation rather than just "multi-task training with a ranking loss" is the specific combination: (1) starting from an embedding model rather than a general-purpose LLM, (2) using PRF-style listwise prompts to inject document context into the embedding computation, and (3) jointly optimizing contrastive and pairwise ranking objectives to produce a single model that performs both tasks through the same scoring mechanism. Each component exists in prior work, but the specific configuration — and its demonstration that a single embedding model can match or exceed dedicated listwise rerankers — is novel and suggests a different development paradigm for ranking systems.

Innovation 4: The Empirical Characterization of PRF Signal Saturation in Embedding-Based Reranking

While the PRF reframing is the paper's conceptual contribution, the empirical characterization of how PRF signals behave in embedding-based reranking is a distinct intellectual contribution with practical implications beyond E2RANK. The paper demonstrates, through controlled experiments, that the benefit of pseudo-relevance feedback in embedding space follows a predictable saturation curve and is bounded by the quality of the feedback documents — patterns that were not previously documented for embedding-based models.

The key evidence is Figure 2, which shows the relationship between the number of documents included in the listwise prompt and the resulting reranking performance on TREC DL19 and DL20. The curve rises sharply from 0 to roughly 10–15 documents, then plateaus and eventually shows slight degradation beyond approximately 20 documents. This pattern reveals three distinct regimes:

The enrichment regime (0–15 documents): Adding more documents to the listwise prompt consistently improves performance because each additional document provides new information about the relevance space — different facets of the query, different document structures, different relevance patterns. The PRF signal grows stronger as the model sees more examples of what constitutes a good document for this query.

The saturation regime (15–20 documents): Performance plateaus because the top-retrieved documents share substantial topical overlap. The 15th document is likely similar in content and relevance signals to the 5th and 10th; adding it provides diminishing new information. The model's enriched query embedding has already captured the essential characteristics of relevant documents, and additional exemplars reinforce rather than expand the representation.

The potential degradation regime (20+ documents): On some datasets, adding more documents can slightly hurt performance. This is consistent with PRF theory: as you go deeper into the retrieval ranking, documents become less likely to be genuinely relevant, and including them in the PRF prompt can introduce noise — the model may incorporate misleading signals from documents that the initial retriever ranked highly but that are not actually relevant.

This characterization matters for two reasons beyond E2RANK's specific design choices. First, it provides a practical guideline for PRF-based methods in embedding space: 15–20 documents appears to be a sweet spot on these datasets, and practitioners can expect diminishing returns or even negative effects beyond this range. Second, it suggests a fundamental bound on the information content of PRF — there is a finite amount of useful relevance signal in the top retrieval results, and once that signal is extracted, adding more documents cannot improve the query representation. This has implications for any method that uses document context to enrich query embeddings, not just listwise reranking.

The relationship between first-stage retriever quality and reranking performance (Appendix D, Table 21) provides converging evidence: stronger retrievers (SPLADE++ED, with higher initial NDCG) yield better reranking results than weaker retrievers (Contriever, with lower initial NDCG), because the PRF signal quality depends on the quality of the feedback documents. This is a falsifiable prediction of the PRF framework — better pseudo-relevance feedback should produce better enrichment — and the data confirms it, lending credibility to the PRF interpretation.

Innovation 5: The Demonstration That Embedding Models Can Benefit from Ranking-Aware Training Without Architectural Modification

A less prominent but intellectually significant contribution is the paper's demonstration that standard contrastive-trained embedding models can be improved — for both retrieval and reranking — by incorporating pairwise ranking signals, without any architectural changes or specialized ranking heads. This challenges a subtle but pervasive assumption in the embedding model literature: that contrastive learning against relevance judgments is the optimal training strategy for retrieval.

The evidence is in Table 4, comparing the "w/ only Stage I" and full E2RANK models on MTEB. Across all three model sizes, Stage II training with the RankNet loss consistently improves retrieval performance:

• E2RANK-0.6B: retrieval improves from 48.07 to 51.74 (+3.67 NDCG)
• E2RANK-4B: retrieval improves from 54.36 to 55.33 (+0.97 NDCG)
• E2RANK-8B: retrieval improves from 55.31 to 56.89 (+1.58 NDCG)

These gains are not explained by additional training data alone — the Stage II training uses only ~87K examples, a tiny fraction of the ~1.5M examples in Stage I, and the contrastive loss remains in the objective throughout Stage II. The improvement must therefore come from the nature of the ranking signal: the RankNet loss teaches the model to produce embedding spaces where relative document ordering is reflected in similarity score differences, which is a stronger constraint than the binary relevance signal in standard contrastive learning. The contrastive loss asks only that positive documents have higher similarity than negatives; the RankNet loss asks that the similarities reflect a full ordering with correct pairwise relationships. This richer training signal appears to produce better-aligned embedding spaces even for the retrieval task alone.

This finding has implications for the broader embedding model literature. It suggests that training strategies developed for learning-to-rank — a subfield of information retrieval that predates modern dense retrieval by decades — may contain untapped value for improving embedding quality. The RankNet loss dates to 2005 (Burges et al.), but its application within contrastive training pipelines for LLM-based embeddings is novel. The paper does not exhaustively explore this connection (it tests only RankNet, not LambdaRank or other listwise losses), but the positive transfer from ranking to retrieval opens a research direction that could benefit the embedding community independently of the reranking application.

The ablation in Table 6 provides an important negative control: removing the listwise prompt ("w/o Listwise in Stage II") while retaining RankNet training causes the reranking benefit to largely disappear (49.93 NDCG vs. 52.09 with listwise prompts on BEIR). This confirms that the ranking signal alone — even when incorporated into the training objective — is insufficient; the model must also learn to use document context (the listwise prompt) during inference to unlock the full reranking capability. The training innovation is thus the combination of ranking-aware training with context-aware inference, not either component alone.

This finding also explains a subtle design choice: the temperature difference between InfoNCE (0.03) and RankNet (0.1). The InfoNCE temperature is tuned to produce sharp discrimination between one positive and many negatives — appropriate for retrieval where the model must identify the few relevant documents among millions. The RankNet temperature is tuned to produce well-calibrated pairwise comparisons among the 16 documents in the training set — appropriate for reranking where the model must produce a smooth ordering of many documents. The fact that both objectives coexist in Stage II and that retrieval improves (not degrades) suggests that the model learns to represent documents in a way that supports both sharp discrimination (for retrieval-scale candidate sets) and fine-grained ordering (for reranking-scale candidate sets). This is a non-trivial representational achievement and a key reason the unified model works.

5. Experimental Analysis

Evaluation Methodology

• Dataset. The paper evaluates on three main benchmarks. BEIR (Thakur et al., 2021), specifically 8 datasets (TREC Covid, NFCorpus, Touché2020, DBPedia, SciFact, Signal1M, TREC News, Robust04) plus TREC DL19 and DL20 (Craswell et al., 2020) for general reranking; the BRIGHT benchmark (Su et al., 2025) covering 12 reasoning-intensive retrieval tasks across StackExchange, coding, and theorem-based domains; and MTEB (Muennighoff et al., 2022) English v1 (56 datasets, 7 task categories) and v2 (41 tasks) for embedding evaluation. For all reranking benchmarks, BM25 serves as the first-stage retriever on BEIR/TREC DL, ReasonIR with GPT-4 reasoned queries on BRIGHT, and the top-100 candidate documents are reranked using NDCG@10 as the metric.

• Base model(s). All experiments use decoder-only instruction-tuned LLMs from the Qwen3 family (Yang et al., 2025) at three scales: 0.6B, 4B, and 8B parameters. The paper argues this family is representative of contemporary open-weight LLMs with strong instruction-following capabilities, and the three-scale sweep enables analysis of how the proposed method's benefits interact with model capacity.

• Metrics. The primary reranking metric is NDCG@10 (Normalized Discounted Cumulative Gain at rank 10), which measures ranking quality in the top positions with a logarithmic discount for lower-ranked documents. For MTEB embedding evaluation, the paper reports task-specific metrics (accuracy for classification, Spearman correlation for STS, NDCG@10 for retrieval and reranking) and an overall average across all 56 (v1) or 41 (v2) tasks. The end-to-end retrieval-reranking pipeline also uses NDCG@10. Latency is measured in seconds per query on a single NVIDIA A100 80G GPU using vLLM (Kwon et al., 2023).

• Baselines. The paper defines three tiers of baselines. (1) Directly comparable fine-tuned listwise rerankers: RankQwen3, which is the same Qwen3 base model fine-tuned on the GPT-4 labeled listwise ranking dataset from Pradeep et al. (2023), using a sliding window strategy of window size 20 and step 10 for inference. This is the most important baseline because it isolates the effect of E2RANK's embedding-based approach versus generation-based reranking at identical model scale. (2) Broader fine-tuned rerankers: monoBERT (340M; Nogueira et al., 2019), monoT5 (3B; Nogueira et al., 2020), RankT5 (3B; Zhuang et al., 2023), ListT5 (3B; Yoon et al., 2024), RankZephyr (Pradeep et al., 2023), and pointwise rerankers trained with the same RankNet loss on the same data as E2RANK (reported in Appendix D, Table 11). (3) Zero-shot listwise rerankers: RankGPT-4o and RankGPT-4o-mini (Sun et al., 2023), and RankQwen3 at 14B and 32B scales. For BRIGHT, additional reasoning reranker baselines include Rank-R1 (7B and 14B; Zhuang et al., 2025), Rank1 (7B; Weller et al., 2025), JudgeRank (8B; Niu et al., 2024), Rearank (7B; Zhang et al., 2025a), ERank (4B and 14B; Cai et al., 2025), and ReasonRank (7B; Liu et al., 2025c). For MTEB embedding evaluation, baselines include Instructor-xl (Su et al., 2022), BGE-large-en-v1.5 (Xiao et al., 2023), GritLM-Mistral-7b-v1 (Muennighoff et al., 2024), E5-Mistral-7b-v1 (Wang et al., 2023), Echo-Mistral-7b-v1 (Springer et al., 2025), and LLM2Vec variants (BehnamGhader et al., 2024).

• Generation budget / compute accounting. The paper does not measure "generations" in the autoregressive sense for E2RANK, since the method produces no generated text. Instead, compute is measured as inference latency (seconds per query) and broken into document encoding time (which can be precomputed offline) and online reranking time (encoding the listwise prompt and computing cosine similarities). For RankQwen3, the latency measurement is end-to-end including both prompt encoding and autoregressive decoding. All efficiency measurements use vLLM on a single NVIDIA A100 80G GPU with the TREC COVID dataset (50 test queries, documents averaging ~350 tokens after Qwen3 tokenization). The paper does not report FLOP counts or parameter-level compute accounting, focusing instead on wall-clock latency as the deployment-relevant metric.

• Cross-validation / statistical protocol. There is no cross-validation or statistical significance testing reported. The test sets are fixed: 50 queries for TREC COVID, varying query counts per BEIR dataset, 500 questions for MATH (not used in this paper), and the standard MTEB test splits. The ablation studies use the Qwen3-0.6B model as the testbed. For difficulty-based strategy selection (not applicable to E2RANK in the same way as the compute-optimal scaling paper), no cross-validation is needed since E2RANK does not adaptively select strategies per query. The paper does note that instructions for evaluation of MTEB are fixed across all models (listed in Appendix C, Table 10) and that different instructions were found to have "a very small impact on performance, at least not statistically significant."

Main Quantitative Results

Reranking Performance on BEIR and TREC DL

The headline result from Table 1 is that E2RANK consistently outperforms the directly comparable RankQwen3 baseline across model sizes, with the gain being most pronounced at the smallest scale. Specifically, E2RANK-0.6B achieves an average NDCG@10 of 52.09 across the 8 BEIR datasets plus DL19/DL20, compared to 48.03 for RankQwen3-0.6B — an improvement of +4.06 NDCG@10. At 4B scale, E2RANK reaches 54.14 versus RankQwen3's 52.38 (+1.76). At 8B scale, E2RANK achieves 54.35 versus 53.39 (+0.96). The diminishing gap with model size is notable: for the smallest model, the embedding-based approach provides substantial gains, while for the largest model, both approaches converge to similar performance, suggesting that larger models can compensate for the inefficiency of generation-based reranking through greater capacity.

The per-dataset breakdown in Table 1 reveals that E2RANK's gains are not uniform. The largest improvements for the 0.6B model occur on TREC News (+8.32 NDCG, from 44.43 to 52.75) and Robust04 (+7.36 NDCG, from 46.31 to 53.67). These are datasets with longer documents and complex relevance patterns where the PRF-enriched query embedding may be capturing nuanced topical signals that the smaller RankQwen3 model fails to express through generation. Conversely, on some datasets, E2RANK slightly underperforms RankQwen3 at larger model sizes: E2RANK-8B trails RankQwen3-8B on DBPedia (43.44 vs. 45.44, -2.00), SciFact (77.49 vs. 78.96, -1.47), and NFCorpus (39.08 vs. 40.05, -0.97). The paper does not analyze these per-dataset regressions, but they may reflect datasets where the fine-grained generation-based ranking (which can explicitly compare documents in the output text) provides benefits that embedding-based scoring cannot fully capture, or where the PRF signal from top-20 documents is less informative.

Table 2 places E2RANK in the context of broader baselines. E2RANK-8B achieves the highest BEIR average (54.35) among all compared models, surpassing fine-tuned listwise rerankers like RankZephyr (51.15) and ListT5 (53.00), and even exceeding zero-shot RankGPT-4o (53.09). This is a striking result: an 8B embedding model, trained only on public data, outperforms GPT-4o — a model estimated to be orders of magnitude larger — on this ranking benchmark. On DL20, E2RANK-8B achieves 71.16, the best among all compared models including RankGPT-4o (69.52). However, on DL19, E2RANK-8B (72.65) trails RankGPT-4o (74.78) and RankGPT-4o-mini (72.36) as well as RankQwen3-14B (74.19), suggesting that the DL19 query set may favor the broader knowledge and reasoning capabilities of much larger models that E2RANK cannot match at 8B scale.

The pointwise reranker baseline (Appendix D, Table 11) trained with the same RankNet loss on the same data as E2RANK achieves substantially lower performance (45.97–48.12 BEIR average across scales), confirming that the listwise prompt mechanism — not just the ranking loss — is essential for the reranking quality. The pointwise model scores each query-document pair independently without document context, and the NDCG gap relative to E2RANK (e.g., 45.97 vs. 52.09 at 0.6B) quantifies the value of the PRF signal.

Reranking Performance on BRIGHT

Table 3 presents results on the reasoning-intensive BRIGHT benchmark. E2RANK-8B achieves an average NDCG@10 of 33.4 across the 12 BRIGHT tasks, surpassing RankQwen3-8B (32.0) and most reasoning-specialized rerankers including Rank-R1-7B (24.1), Rank1-7B (24.3), Rearank-7B (27.5), and JudgeRank-8B (20.2). It is slightly outperformed by ERank-14B (31.8) and ReasonRank-7B (35.7) — the latter being trained on synthetic reasoning data specifically designed for BRIGHT-style tasks.

The per-task pattern reveals that E2RANK's strengths and weaknesses on BRIGHT are task-dependent. It performs well on several StackExchange tasks (Biology: 49.2, Economics: 47.2, Earth Science: 32.3) and on LeetCode (38.2), but struggles on Pony (10.6) and Art of Problem Solving (8.2) relative to some baselines (ReasonRank reaches 23.2 on Pony and 7.7 on AoPS). This suggests that E2RANK's PRF mechanism is effective when the top-retrieved documents contain useful comparative signals, but when the reasoning required is fundamentally about mathematical or logical structure (AoPS, TheoremQA), the embedding-based approach may not capture the necessary inference patterns that generation-based reasoning rerankers can articulate.

A notable finding is that E2RANK-0.6B (31.0 average) substantially outperforms RankQwen3-0.6B (29.1) on BRIGHT, and even surpasses many larger reasoning rerankers (Rank-R1-14B at 29.7). This reverses the trend from BEIR where gains were largest at small scale — on reasoning-intensive tasks, the embedding-based approach appears to extract more value from limited model capacity than generation-based methods, perhaps because the PRF mechanism leverages the model's existing embedding quality (which is non-trivial even at 0.6B) rather than requiring the model to generate coherent reasoning chains (which is capacity-intensive).

Reranking Efficiency

Figure 1c and Appendix D Tables 12-13 present the latency measurements on the TREC COVID dataset. For E2RANK-8B, the total reranking latency per query is 3.40 seconds, broken into 2.76 seconds for document encoding (which can be precomputed offline) and 0.64 seconds for encoding the listwise prompt and computing cosine similarities. For RankQwen3-8B using sliding window inference, the total latency is 16.93 seconds per query. The online-only latency (0.64 seconds for E2RANK-8B) represents approximately a 26× reduction compared to RankQwen3-8B. Even including document encoding, the total E2RANK-8B latency (3.40s) is 5× faster than RankQwen3-8B (16.93s) and is actually faster than RankQwen3-0.6B (4.58s), while achieving substantially higher accuracy (54.35 vs. 48.03 NDCG on BEIR average). This efficiency-accuracy tradeoff is the paper's strongest practical argument: E2RANK simultaneously improves both metrics relative to the generation-based approach at matched model size.

The efficiency advantage scales with model size. At 0.6B, E2RANK (0.63s total, 0.13s online) is 7.3× faster than RankQwen3 (4.58s). At 4B, E2RANK (2.17s total, 0.43s online) is 5.2× faster than RankQwen3 (11.25s). The speedup ratio decreases with model size because the embedding extraction cost grows more slowly than generation cost as model capacity increases — generation requires processing both input and output tokens, while embedding extraction uses only the input.

Embedding Performance on MTEB

Table 4 presents MTEB results, establishing that E2RANK maintains competitive embedding quality despite its reranking-focused Stage II training. E2RANK-8B achieves an average score of 65.03 across 56 tasks, slightly ahead of LLM2Vec-Meta-LLaMA-3-8B (65.01) and Echo-Mistral-7b-v1 (64.68). The model performs well on retrieval tasks (56.89, the highest among compared models), classification tasks (76.81), and STS tasks (84.52), but is weaker on clustering (44.75, below BGE-large's 46.08) and summarization (30.23, below Instructor-xl's 32.32).

The comparison between Stage I-only and full E2RANK within each model size quantifies the effect of ranking-aware training on embedding quality. For the 8B model, Stage II training improves retrieval from 55.31 to 56.89 (+1.58 NDCG), classification from 75.69 to 76.81, and STS from 83.23 to 84.52, while slightly reducing clustering from 45.84 to 44.75 and reranking from 55.73 to 59.58. The overall average improves from 64.26 to 65.03. This positive transfer from ranking training to embedding quality is consistent across all three model sizes: +1.20 for 0.6B, +0.86 for 4B, and +0.77 for 8B, with the effect diminishing as the base model becomes stronger. The paper does not provide a mechanistic explanation for why ranking training improves retrieval, but one likely reason is that the RankNet loss encourages the embedding space to reflect fine-grained relevance differences that contrastive learning with binary labels misses.

End-to-End Unified Retrieval and Reranking

Table 5 evaluates using E2RANK as the sole model for both retrieval (first-stage) and reranking (second-stage) in a complete search pipeline. For E2RANK-8B on BEIR, retrieval alone achieves 53.39 NDCG@10 and adding reranking raises this to 55.08 — a gain of +1.69 NDCG. On BRIGHT, retrieval achieves 25.09 and reranking improves this to 31.00 — a more substantial gain of +5.91 NDCG. The larger reranking gain on BRIGHT is consistent with the intuition that reasoning-intensive tasks benefit more from the comparative signals in the listwise prompt, since the initial retrieval is more likely to return documents that are topically relevant but not necessarily the correct answer, and the PRF mechanism helps distinguish truly relevant documents from misleading ones.

The end-to-end results also confirm that E2RANK's retrieval quality alone (without reranking) is competitive: on BEIR, E2RANK-8B retrieval at 53.39 is comparable to the best embedding models in Table 4. This validates the paper's claim that a single model can serve both stages without sacrificing retrieval quality for reranking capability.

Ablation Studies and Robustness Checks

Table 6 (Section 4.5) presents the core ablation study using the Qwen3-0.6B model, evaluated on DL20, BEIR, BRIGHT, and MTEB v2. Each ablation removes or modifies one component of the training pipeline.

Removing Stage I entirely ("w/o Stage I"): Training directly with the Stage II multi-task objective without the contrastive learning foundation causes reranking performance to drop modestly on BEIR (52.09 → 51.33) and BRIGHT (30.96 → 30.66), but causes a sharp decline in MTEB embedding quality (63.41 → 60.61). This confirms that the contrastive pretraining is essential for the model's general embedding capability — the ranking objective alone, trained on only 87K examples, is insufficient to learn a good embedding space from scratch. The reranking drop being modest rather than catastrophic suggests that the RankNet loss can partially compensate for a weaker embedding foundation when the model is being used for reranking (where the listwise prompt provides additional context), but not for pure retrieval.

Removing InfoNCE from Stage II ("w/o InfoNCE in Stage II"): Training Stage II with only the RankNet loss (no contrastive objective) yields mixed results. On BEIR, reranking performance remains essentially unchanged (52.17 vs. 52.09) and on DL20 it improves slightly (69.11 vs. 70.15). On BRIGHT, reranking declines from 30.96 to 29.99. On MTEB v2, embedding quality drops substantially from 63.41 to 61.92. This suggests that the contrastive loss in Stage II is more important for preserving embedding quality than for maintaining reranking performance — the RankNet loss alone can sustain reranking ability (perhaps because the listwise prompt provides enough context to compensate for some embedding degradation), but the model's retrieval capability erodes without the contrastive objective actively maintaining the query-document alignment.

Using only Stage I ("w/ only Stage I"): This is the standard embedding model without any ranking training, evaluated using query-only embeddings (no listwise prompt). Reranking performance collapses: on BEIR from 52.09 to 46.31, on BRIGHT from 30.96 to 15.30, and on DL20 from 70.15 to 63.55. This is the expected result — an embedding model trained only for retrieval has no mechanism to use document context for comparative scoring. The MTEB v2 score drops from 63.41 to 62.40, confirming that even without the reranking capability, the Stage I model has strong embedding quality (and Stage II further improves it).

Removing RankNet from Stage II ("w/o RankNet in Stage II"): Training Stage II with only InfoNCE loss (contrastive learning, no ranking objective) severely degrades reranking: BEIR drops from 52.09 to 49.24, BRIGHT drops from 30.96 to 22.40, DL20 drops from 70.15 to 66.50. MTEB v2 is relatively unaffected (63.41 → 63.31, a tiny decline). This is the complement of the previous ablation — the RankNet loss is essential for reranking but contributes little to pure embedding quality. The reranking collapse confirms that the model's ability to perform comparative scoring comes specifically from the pairwise ranking objective, not from simply seeing more contrastive training data in Stage II.

Removing the listwise prompt at inference ("w/o Listwise in Stage II"): This ablation is critical for understanding the mechanism. The model is trained with the full Stage II objective (InfoNCE + RankNet), but at evaluation time, documents are scored using query-only embeddings (no listwise prompt), effectively using the model as a standard retriever. Reranking performance drops dramatically: BEIR falls from 52.09 to 49.93, BRIGHT drops from 30.96 to 22.69, DL20 drops from 70.15 to 66.29. The fact that reranking performance collapses without the listwise prompt — even though the model was trained with the RankNet loss — demonstrates that the reranking capability is not "baked into" the model weights in a way that activates during standard query encoding. The model must see the document context (the listwise prompt) to produce the PRF-enriched query embedding that enables comparative scoring. This confirms the paper's central mechanism: it is the combination of ranking-aware training AND context-aware inference that produces the gains, not either alone.

The ablation also reveals that MTEB v2 performance is slightly higher without the listwise prompt (63.66 vs. 63.41), likely because MTEB evaluation uses query-only embeddings regardless (the listwise prompt is not applicable for tasks like classification or clustering), and the model may be slightly distracted by having been trained to expect document context that is absent during pure embedding tasks. The difference is small (+0.25), suggesting robustness.

Influence of number of input documents in the listwise prompt (Figure 2): This ablation sweeps the number of documents included in the listwise prompt from 0 to 100 on DL19 and DL20. Both datasets show a rapid improvement from 0 to ~10-15 documents, then a plateau with slight degradation beyond ~20 documents. On DL19, NDCG@10 rises from approximately 65 at 0 documents to roughly 71 at 20 documents, then fluctuates between 70-71 up to 100 documents. On DL20, the curve is similar: from approximately 65 at 0 to roughly 70 at 15, plateauing there. The paper interprets the degradation at higher document counts (visible as small dips on DL19 at 60+ documents) as noise from lower-ranked documents providing misleading PRF signals. This ablation validates the design choice of using top-20 documents in the listwise prompt — it captures nearly all the benefit while avoiding the computational cost of longer prompts.

Score distribution analysis (Figure 3): Comparing the similarity score distributions between listwise prompts (with 20 documents) and query-only prompts reveals a sharpening effect. With the listwise prompt, top-ranked documents receive consistently higher similarity scores, and the score curve decays more steeply as rank decreases. With query-only, the score distribution is flatter — top documents are less clearly separated from lower-ranked ones. Quantitatively, the similarity for rank-1 documents is approximately 0.75 with listwise versus 0.55 with query-only; by rank-20, both converge to similar scores. This provides a mechanistic explanation for the reranking improvement: the PRF-enriched query embedding more confidently discriminates between relevant and irrelevant documents, producing sharper score gradients that translate to better NDCG.

Influence of different first-stage retrievers (Appendix D, Table 21): This robustness check evaluates E2RANK and RankQwen3 when the initial candidate set comes from different retrievers: BGE-base, Contriever, SPLADE++ED, and Qwen3-Embedding-0.6B. E2RANK consistently outperforms RankQwen3 across all retrievers and model sizes, with the reranking gain being larger when the initial retriever is stronger. For example, using SPLADE++ED (the strongest retriever with DL19 NDCG of 73.08), E2RANK-8B achieves 77.37 on DL19 versus RankQwen3-8B's 74.61 (+2.76). Using Contriever (the weakest, DL19 NDCG of 62.02), E2RANK-8B achieves 73.77 versus RankQwen3-8B's 72.62 (+1.15). This confirms the PRF interpretation: better first-stage retrieval produces better pseudo-relevance feedback, which leads to better reranking.

Full ranking label quality (Appendix B, Figure 5): The paper measures the "accuracy" of Qwen3-32B's generated ranking labels against the dataset's original golden positives — the frequency with which the LLM places the dataset's positive document at rank 1. Agreement rates vary from 54.3% (MS MARCO) to 91.3% (HotpotQA). The paper acknowledges this noise but does not ablate its effect — there is no comparison of training with Qwen3-32B labels versus dataset ground-truth labels (which would require a different label format since datasets provide only binary relevance). This is a notable omission: if the LLM labels are noisy on MS MARCO (nearly half the time the LLM disagrees with the dataset about which document is most relevant), the training signal is substantially corrupted, yet the model still learns effective reranking. This could indicate that the pairwise ranking objective is robust to label noise — it only needs relative orderings to be mostly correct — or that the LLM labels capture relevance dimensions that the dataset creators didn't annotate but that are genuinely useful for ranking.

Critical Assessment

The experiments collectively support the paper's central claim — that a single embedding model can unify retrieval and listwise reranking with competitive accuracy and substantially lower latency than generation-based alternatives — but several boundaries on this claim deserve scrutiny.

Does E2RANK genuinely "unify" retrieval and reranking, or does it add reranking capability to an embedding model that then serves as two separate inference modes? The paper's usage of "unified" refers to the model weights being shared and the scoring function (cosine similarity) being identical for both stages. But the inference procedure differs: retrieval uses a query-only prompt, reranking uses a listwise prompt. This is a unified model but not a unified inference mechanism — the model must be invoked differently for the two tasks. The end-to-end results (Table 5) show this two-mode approach works, but it is less "unified" than the name suggests. A truly unified approach would use the same input format for both stages (perhaps varying only the number of candidate documents) and have the model output scores directly. The paper's approach is closer to a "dual-mode" model than a fully unified one.

Is the efficiency advantage as large as claimed when document encoding is not precomputable? The paper's headline efficiency numbers (5× speedup at 8B, 26× for online-only) assume document embeddings are precomputed and cached. For applications where documents are dynamic or the candidate set changes per query (e.g., in RAG where documents are retrieved from a constantly updating corpus), the document encoding cost (2.76 seconds for E2RANK-8B) cannot be amortized, reducing the total latency advantage to 16.93s vs. 3.40s — still 5× but not the 26× implied by the online-only figure. The paper is transparent about this breakdown (Tables 12-13), but the abstract and introduction emphasize the larger speedup without always qualifying the precomputation assumption.

How much does the reranking performance depend on the specific Qwen3 family? All experiments use Qwen3 as the base model. The paper argues this family is "representative of contemporary LLMs," but the Qwen3 models were instruction-tuned with specific chat formats and reasoning capabilities that may interact with the listwise prompt mechanism in ways that don't generalize. For example, the chat template tokens (<im_start>user, <im_end>, <im_start>assistant) are Qwen3-specific; other model families use different formats. The paper does not evaluate on Llama, Mistral, or other popular backbones. This single-model-family evaluation limits confidence in the method's generality, though the consistency of gains across three scales within the family is reassuring.

Does the RankNet loss genuinely teach comparative reasoning, or does it simply teach the model to assign higher similarity to documents that appear earlier in the listwise prompt? The ablation showing that removing the listwise prompt at inference causes reranking to collapse (Table 6, "w/o Listwise") rules out the possibility that the model has learned a query-only scoring function that happens to align with rankings. But it does not rule out simpler heuristics — for example, the model might learn to assign higher scores to documents that appear at specific positions in the prompt or that share more tokens with the instruction text. The score distribution analysis (Figure 3) partially addresses this by showing discriminative scoring, but a controlled experiment where document order is randomized in the prompt (to test for position bias) or where documents are replaced with topic-matched but irrelevant alternatives (to test for content-based scoring) would provide stronger evidence of genuine comparative reasoning.

Are the BRIGHT results genuinely competitive with reasoning-specialized rerankers? E2RANK-8B achieves 33.4 NDCG on BRIGHT, which is stronger than most baselines but notably below ReasonRank-7B (35.7). However, ReasonRank was trained on synthetic reasoning data specifically designed for BRIGHT-style tasks, giving it a training data advantage. The paper does not train ReasonRank on the same data as E2RANK, making the comparison somewhat apples-to-oranges. A fairer comparison would control for training data by training E2RANK on the same reasoning data or training ReasonRank on the E2RANK training data. The paper's claim of "competitive performance" is accurate given the data constraint, but the gap to ReasonRank (the strongest baseline) is not small on several tasks (e.g., Psychology: 44.7 for E2RANK vs. 56.7 for ReasonRank; Robotics: 10.6 vs. 23.2).

The ablation on listwise prompt size (Figure 2) uses only DL19 and DL20. These are relatively homogeneous newswire datasets. The saturation point of ~20 documents might differ substantially on other BEIR datasets (e.g., NFCorpus has very short documents, while SciFact has longer scientific abstracts) or on BRIGHT. The paper uses the 20-document cutoff across all benchmarks without per-dataset validation, which may leave performance on the table for datasets where PRF signals benefit from more documents or may slightly hurt performance where saturation occurs earlier.

Missing ablation: the chat template versus plain instruction format. The paper notes that chat templates are applied for listwise prompts but not for standard embedding. This is a confound in the comparison between retrieval and reranking modes — the reranking improvement might partially stem from the chat template signaling instruction-following behavior, not just from the document context. An ablation comparing listwise prompts with and without chat templates would isolate this effect.

Missing baseline: using the same listwise prompt with a cross-encoder scoring head. The paper compares against pointwise cross-encoders and generation-based listwise rerankers, but not against a model that encodes the listwise prompt and then uses a learned scoring head (e.g., a small MLP on top of the [EOS] embedding) to score each document, rather than cosine similarity. This would test whether the cosine similarity constraint — which ensures retrieval-reranking unification — imposes a performance penalty relative to a more expressive scoring function. If such a model substantially outperformed E2RANK, it would suggest that unification comes at a cost to reranking quality that the paper does not measure.

The efficiency comparison with RankQwen3 uses a sliding window strategy (window 20, step 10) for the baseline. This is a necessary design for generation-based reranking of 100 documents (since encoding all 100 in one prompt would exceed context limits or be prohibitively slow), but it means the baseline is not using the same information as E2RANK. E2RANK sees 20 documents in its prompt and scores all 100; RankQwen3 sees 20 documents at a time in overlapping windows. A full-ranking RankQwen3 (all 100 documents in one prompt, feasible at 8B with long-context models) would be a stronger baseline but is not reported — the paper notes in Appendix D that "full ranking is less effective" without providing numbers.

The MTEB evaluation uses the full E2RANK model without listwise prompts, effectively evaluating the embedding quality of the model when used in retrieval mode. This is appropriate for showing that Stage II training doesn't destroy embedding quality, but it does not test whether the reranking capability transfers to embedding benchmark tasks that could benefit from document context — for example, the reranking subtask in MTEB. The paper reports the reranking MTEB score as 59.58 for E2RANK-8B (Table 4), but it's unclear whether this uses the listwise prompt or query-only encoding.

These limitations do not undermine the paper's core contributions, but they bound the claims in important ways. The method clearly works on Qwen3-family models on BEIR and BRIGHT; its generality to other model families, its sensitivity to prompt format, and the degree to which the cosine similarity constraint limits reranking quality remain open questions.

6. Limitations and Trade-offs

6.1 Single Model Family Evaluation Limits Generality Claims

The assumption or constraint. All experiments — reranking on BEIR/BRIGHT, embedding on MTEB, efficiency measurements, and ablation studies — use decoder-only models from the Qwen3 family (0.6B, 4B, 8B) as the backbone. The paper states that it "believe[s] this model is representative of the capabilities of many contemporary LLMs" (Section 4.1), but provides no evaluation on Llama, Mistral, Gemma, or other widely deployed embedding backbones. The chat template format, instruction-tuning procedure, and pretraining data distribution of Qwen3 may all interact with E2RANK's listwise prompt mechanism in ways that do not transfer.

The consequence. A practitioner using a different model family (e.g., fine-tuning Llama-3 embeddings for a production RAG pipeline) cannot assume E2RANK's reported gains — +4.06 NDCG over generation-based reranking at 0.6B, 5× latency reduction at 8B — will replicate. The chat template tokens (<|im_start|>user, <|im_end|>, <|im_start|>assistant) are Qwen3-specific; other models use entirely different conversation formats (Llama's [INST] tags, Mistral's tokens). The method's core mechanism — using the listwise prompt as a PRF-enriched query — may depend on the model's ability to attend across instruction, document, and query segments, which could vary with pretraining recipe and architecture details that differ across model families. Additionally, Qwen3's instruction tuning may be particularly well-suited to listwise prompt interpretation in ways that other models' tuning is not, making replication uncertain.

What evidence exists in the paper. There is none. The paper provides zero cross-model-family experiments. The consistency of gains across three scales within Qwen3 (0.6B, 4B, 8B) demonstrates that the method is robust to model size within the family, but this is weak evidence for cross-family generalization — models within a family share architecture, training data, tokenizer, and tuning procedure.

Mitigation status. Not addressed. The paper does not acknowledge this as a limitation in Section 5 (Conclusion) or elsewhere, and does not suggest cross-family evaluation as future work. The "representative" claim in Section 4.1 is asserted without justification or caveat.

6.2 The Gap to Reasoning-Specialized Rerankers on Hard Tasks Is Substantial

The assumption or constraint. E2RANK treats listwise reranking as an embedding-space operation: the comparative signals from candidate documents are compressed into a single query vector, and ranking reduces to cosine similarity. This approach inherently limits the model's ability to perform explicit multi-step reasoning — comparing evidence across documents, resolving contradictions, applying logical constraints — because all such reasoning must be "compiled" into the fixed-dimensional embedding representation rather than articulated as a reasoning chain.

The consequence. On the reasoning-intensive BRIGHT benchmark, E2RANK-8B achieves 33.4 NDCG average (Table 3), which is competitive with most baselines but trails ReasonRank-7B (35.7) by a non-trivial margin of 2.3 NDCG points. The gap is particularly large on specific tasks: Psychology (44.7 for E2RANK vs. 56.7 for ReasonRank, a 12-point gap), Robotics (10.6 vs. 23.2), and TheoremQA (33.4 vs. 41.8). ReasonRank is trained on synthetic reasoning data specifically designed for BRIGHT-style tasks, giving it a training data advantage, but the magnitude and pattern of the gap suggest a capability boundary: when ranking requires explicit reasoning about document content (not just semantic similarity assessment), the embedding-based approach may fundamentally underperform generation-based or RL-trained reasoning rerankers that can articulate step-by-step comparisons. The paper acknowledges that E2RANK operates "without any RL or reasoning process" (Section 4.2), framing this as a feature (efficiency) rather than a limitation, but it is both.

What evidence exists in the paper. Table 3 shows E2RANK-8B outperforming most reasoning rerankers (Rank-R1, Rank1, Rearank, JudgeRank) but consistently trailing ReasonRank. The per-task BRIGHT breakdown reveals that E2RANK's underperformance relative to ReasonRank is concentrated on the hardest reasoning categories (Theorem-based tasks average: 33.4 for E2RANK-8B vs. RankQwen3-8B's 32.0, a small gap, but ReasonRank achieves 41.8 on these tasks). The paper also reports (Appendix D, Table 22) that when using BM25 as first-stage retriever (a weaker starting point), E2RANK-8B achieves 22.5 NDCG versus ReasonRank's 35.7 — a much larger gap — suggesting that E2RANK's performance on reasoning tasks degrades more sharply when the initial retrieval quality is poor, consistent with the PRF mechanism's dependence on feedback document quality.

Mitigation status. Partially acknowledged. The paper notes that ReasonRank is "trained on synthetic reasoning data" (Table 3 caption), implicitly attributing the gap to data rather than architecture. However, the paper does not train E2RANK on similar reasoning data to test whether more data can close the gap or whether the embedding bottleneck is fundamental. This is a testable hypothesis that the paper leaves unexplored, making it unclear whether E2RANK can reach reasoning-reranker parity with better training data or whether there is a hard ceiling imposed by the embedding-based scoring mechanism.

6.3 Difficulty Estimation and Adaptive Strategy Selection Are Not Addressed

The assumption or constraint. E2RANK applies the same fixed strategy to every query: encode the top-20 documents from first-stage retrieval into the listwise prompt and rerank all 100 candidates using cosine similarity. The paper never examines whether this uniform allocation is optimal or whether different queries — easy vs. hard, short vs. long, factoid vs. reasoning — would benefit from different numbers of input documents, different prompt formats, or even a fallback to generation-based reranking when embedding-based scoring fails.

The consequence. The paper reports average performance across entire benchmark query sets, potentially masking substantial per-query variance. On easy queries where the initial retriever already ranks the correct document highly, the listwise prompt may provide minimal benefit while still incurring its computational cost (encoding 20 documents into the prompt). On hard queries where the top-20 documents contain no truly relevant exemplars, the PRF mechanism may actively mislead — the enriched query embedding incorporates noise from pseudo-relevant documents that are not actually relevant, potentially degrading rather than improving the ranking. The paper's analysis of PRF signal saturation (Figure 2) shows that the marginal benefit of additional documents plateaus at ~20 on average, but this average hides per-query heterogeneity. A query with clear topical focus might saturate at 5 documents; an ambiguous or multi-faceted query might benefit from 50. The fixed 20-document strategy leaves potential efficiency gains (using fewer documents when sufficient) and potential accuracy gains (using more documents when needed) on the table.

What evidence exists in the paper. The score distribution analysis (Figure 3) shows that the listwise prompt sharpens score discrimination on average, but there is no per-query analysis showing variance in this effect. The ablation on input document count (Figure 2) reports only aggregate NDCG across all DL19 and DL20 queries, not per-query or per-difficulty-stratum breakdowns. The paper does not bucket queries by initial retrieval quality or query type to test whether the optimal document count varies systematically. The end-to-end results (Table 5) show that reranking improves over retrieval on average, but do not report what fraction of queries are improved vs. degraded vs. unchanged.

Mitigation status. Not addressed. The paper does not discuss adaptive strategy selection, difficulty estimation, or per-query variance. This is a notable gap given that the paper's central reframing — treating the listwise prompt as PRF — immediately suggests that PRF quality should depend on initial retrieval quality, which varies per query. The paper's conclusion that a single model can serve as a "unified retrieval-reranking engine" would be stronger with evidence that the non-adaptive strategy is robust to query heterogeneity or with a proposed mechanism for adaptive allocation.

6.4 No Measurement of Training Compute Cost or Data Efficiency Tradeoffs

The assumption or constraint. The paper reports inference-time efficiency gains (5× latency reduction over RankQwen3 at 8B) but provides no accounting of the training cost required to achieve these gains. Stage I requires training on ~1.5M query-document pairs with full-parameter fine-tuning of an 8B model on 8× A100 GPUs for 1 epoch. Stage II requires training on ~87K annotated ranking examples (each with 16 documents and LLM-generated labels from Qwen3-32B) for ~700 steps. The paper never reports GPU-hours, total FLOPs, or training time for either stage.

The consequence. A practitioner deciding between E2RANK and an alternative (e.g., deploying a larger zero-shot reranker like RankGPT-4o-mini via API, or fine-tuning RankZephyr on their own data) cannot evaluate the total cost of ownership. The inference efficiency gains reported in Figures 1 and Tables 12-13 represent operational savings, but these must be amortized against the upfront training investment. If Stage II training on ~87K LLM-labeled examples requires significant compute (8× A100 GPUs is substantial, and generating labels with Qwen3-32B likely incurred additional inference cost), the crossover point where E2RANK becomes cheaper than alternatives depends on query volume. For low-volume applications (thousands of queries), the training cost may dominate; for high-volume applications (millions of queries), the inference savings will eventually recoup the training investment. The paper provides no data to estimate this crossover.

Additionally, the Stage II training data — ~87K examples with 16 documents each, labeled by Qwen3-32B — represents a non-trivial resource that may not be available to all practitioners. The paper does not ablate how performance scales with the amount of Stage II training data (e.g., would 10K examples suffice? 50K?), making it difficult to assess the minimum viable data budget for replicating the approach.

What evidence exists in the paper. None. Section 4.1 reports hardware configuration (8× A100 80G) and training duration (1 epoch, ~700 steps) but does not convert these to total GPU-hours, cost estimates, or comparisons with baseline training costs. Appendix C provides hyperparameter details but no timing measurements. The paper does not report the cost of generating ranking labels with Qwen3-32B (number of API calls or inference hours).

Mitigation status. Not addressed. The paper focuses exclusively on inference-time efficiency and treats training as a one-time cost outside the scope of evaluation. This is common in the reranking literature but is a practical limitation for deployment decisions. The paper does not suggest training cost measurement or data scaling experiments as future work.

6.5 The BRIGHT Benchmark Comparison Uses Different First-Stage Retrievers Than the BEIR Comparison

The assumption or constraint. For BEIR and TREC DL evaluations (Tables 1, 2, 11), E2RANK uses BM25 as the first-stage retriever to produce the top-100 candidates — a standard, well-characterized sparse retrieval baseline. For BRIGHT evaluation (Table 3), the paper switches to ReasonIR with GPT-4 reasoned queries as the first-stage retriever, which is a substantially stronger and more sophisticated retrieval mechanism that generates reasoning-augmented queries before retrieval.

The consequence. The reranking performance on BRIGHT is measured relative to a different (and stronger) starting point than the BEIR performance, making cross-benchmark comparisons of E2RANK's reranking improvement misleading. The paper reports that E2RANK-8B achieves 33.4 NDCG on BRIGHT after reranking, but the ReasonIR baseline (Table 3, top row) achieves already strong retrieval performance — for example, 43.5 on Biology, 30.6 on StackExchange, 36.7 on AoPS. The incremental gain from E2RANK reranking over this strong baseline may be smaller than what would be achieved over a weaker retriever (or vice versa), and the paper provides no BM25 baseline for BRIGHT in the main results (though Appendix D, Table 22 provides a partial BM25 comparison showing substantially lower absolute performance: 22.5 NDCG for E2RANK-8B with BM25 vs. 33.4 with ReasonIR). The choice of ReasonIR makes E2RANK appear stronger on BRIGHT in absolute terms, while making the reranking improvement harder to interpret — a reader cannot easily compare the BRIGHT results to the BEIR results because the first-stage retrieval quality is incommensurate.

Furthermore, this choice interacts with the PRF mechanism in ways the paper does not analyze. ReasonIR uses GPT-4 to generate a reasoning-augmented query, which likely produces top-20 documents that are more precisely targeted to the query's reasoning requirements than BM25 would. This is better PRF — the pseudo-relevant documents fed into the listwise prompt are more likely to be genuinely relevant — so E2RANK's reranking quality on BRIGHT may partially reflect the quality of ReasonIR's retrieval rather than E2RANK's intrinsic reranking capability. A practitioner deploying E2RANK with a weaker retriever (e.g., standard dense retrieval without query rewriting) may not achieve the reported BRIGHT numbers.

What evidence exists in the paper. Table 3 reports the ReasonIR first-stage baseline. Appendix D, Table 22 provides a parallel evaluation using BM25 as first-stage retriever on BRIGHT, where E2RANK-8B achieves 22.5 NDCG (vs. 33.4 with ReasonIR) — a direct demonstration of how much the first-stage retriever matters. However, this comparison is relegated to an appendix and is not discussed in the main text. The paper does not report what fraction of BRIGHT queries are improved vs. unchanged vs. degraded by reranking relative to ReasonIR, nor does it analyze whether E2RANK's reranking improvement over ReasonIR correlates with ReasonIR's retrieval quality per query.

Mitigation status. Partially addressed by Appendix D, Table 22, which shows BRIGHT results with BM25 retrieval and original (non-reasoned) queries. However, these numbers are not integrated into the main narrative, and the paper does not discuss the sensitivity of BRIGHT performance to first-stage retriever choice. The main text (Section 4.2) presents the ReasonIR-based results without caveat. A practitioner reading only the main results would reasonably assume E2RANK achieves 33.4 NDCG on BRIGHT under standard retrieval conditions, which is not the case (standard BM25 yields 22.5).

6.6 The Ablation on Listwise Prompt Size Is Evaluated on Only Two Datasets

The assumption or constraint. The key practical design choice — feeding exactly 20 documents into the listwise prompt to rerank 100 candidates — is justified by Figure 2, which sweeps the number of input documents from 0 to 100 on TREC DL19 and DL20. The paper interprets the plateau at ~20 documents as evidence that "the marginal benefit of adding more feedback signals diminishes once the prompt already captures sufficient relevance context" (Section 4.6), and applies this 20-document cutoff across all BEIR datasets, BRIGHT, and the end-to-end retrieval experiments.

The consequence. The optimal number of input documents for PRF likely depends on dataset characteristics — document length, topical diversity, query specificity, and initial retrieval quality. DL19 and DL20 are newswire datasets with documents of moderate length and queries that are well-formed information needs. On other BEIR datasets, the saturation point may differ substantially:

• NFCorpus has very short documents (biomedical abstracts averaging a few sentences). The information content per document is low, so more documents may be needed to accumulate sufficient PRF signal, and the plateau may occur later.
• SciFact has longer scientific abstracts where documents carry more self-contained information; fewer documents may saturate the PRF signal.
• Touché2020 involves argumentative documents where relevance is subjective and multi-faceted; the PRF signal may be noisier and require more documents to stabilize.
• BRIGHT involves reasoning-intensive queries where the relevant documents may look very different from each other (different proof approaches for the same theorem); the PRF signal from top-20 documents may not adequately capture the diversity of valid relevance patterns.

Applying a uniform 20-document cutoff across all these settings — without per-dataset validation — may leave performance on the table for some datasets (where more documents would help) and may slightly hurt performance on others (where fewer documents would suffice and the extra documents add noise or computational cost). The paper's efficiency analysis (Tables 12-13) is based on the 20-document prompt, so the latency numbers are correct for the evaluated configuration, but they do not reflect what latency would be at alternative configurations that might achieve better accuracy.

What evidence exists in the paper. Figure 2 shows the saturation pattern for DL19 and DL20 only. There is no equivalent analysis for any BEIR dataset, BRIGHT dataset, or MTEB task. The paper does not report whether performance on, for example, NFCorpus or SciFact changes if the number of input documents is varied. The per-dataset results in Table 1 show that E2RANK's improvement over RankQwen3 varies substantially by dataset (from -2.00 on DBPedia at 8B to +10.50 on Touché at 4B), which may partially reflect suboptimal choice of input document count for some datasets, but this hypothesis is untested.

Mitigation status. Not addressed. The paper does not discuss per-dataset tuning of the input document count, does not report sensitivity analyses beyond DL19/DL20, and does not suggest that the 20-document cutoff should be validated or tuned for new datasets. The paper treats the Figure 2 result as sufficient justification for the universal 20-document default. A practitioner applying E2RANK to a new dataset or domain would need to perform their own hyperparameter sweep (which requires labeled evaluation data) to determine the optimal document count — guidance the paper does not provide.

7. Implications and Future Directions

How This Work Changes the Landscape

E2RANK changes the landscape of information retrieval by demonstrating that the boundary between embedding-based retrieval and listwise reranking is an artifact of how models are trained, not a fundamental architectural constraint. This is not a paradigm shift in the sense of introducing a new model class or learning algorithm — the components (contrastive learning, RankNet loss, PRF) all predate this paper. Rather, it is a reconfiguration that resolves a persistent tension in production IR systems by showing that a single embedding model, trained with the right multi-task objective and prompted with document context, can match or exceed the ranking quality of generation-based listwise rerankers while operating at a fraction of their inference cost.

The reframing that enables this — treating the listwise prompt as pseudo-relevance feedback — is the paper's most consequential conceptual move. Prior to this work, the field implicitly assumed that listwise reranking required either autoregressive generation (RankGPT) or some form of full-prompt LLM inference (logit-based or attention-based methods). E2RANK demonstrates that the comparative reasoning that makes listwise reranking powerful can be compressed into a single embedding vector — the PRF-enriched query representation — and that simple cosine similarity against this vector produces rankings competitive with much more expensive mechanisms. This insight decouples "thinking about documents comparatively" from "generating text about documents," and is likely to influence future reranker designs regardless of whether they adopt E2RANK's specific architecture.

The work also resolves the apparent contradiction between the high accuracy and high latency of LLM-based listwise rerankers. The contradiction was never that listwise methods were inaccurate — they were state-of-the-art — but that their computational cost made them impractical for many deployment scenarios. E2RANK dissolves this tension by showing that the ranking quality comes from the interaction between query and documents in context, not from the generation process, and that this interaction can be captured through embedding-space operations. The practical consequence is a reranker that achieves competitive BEIR performance (54.35 NDCG for 8B, surpassing GPT-4o at 53.09) while running 5× faster than the generation-based equivalent and 26× faster when document embeddings are precomputed.

This work also makes generation-free reranking a more attractive research direction. Prior to E2RANK, efficiency-oriented reranking work operated within the constraint of reducing generation cost — compressing inputs, constraining output vocabularies, or distilling to smaller models. E2RANK shows that eliminating generation entirely is not only feasible but can improve both accuracy and efficiency simultaneously (E2RANK-0.6B outperforms RankQwen3-0.6B by +4.06 NDCG while being 7.3× faster). This suggests that the research effort invested in making generation-based rerankers faster may be better directed toward making embedding-based rerankers more expressive — a methodological pivot that the paper implicitly advocates.

Conversely, the work makes purely generation-based listwise reranking without embedding integration a less attractive direction for all but the most reasoning-intensive tasks. If an 8B embedding model can match GPT-4o on BEIR reranking at a tiny fraction of the inference cost, the case for deployment-scale generation-based reranking narrows to scenarios where the reasoning demands genuinely exceed what an embedding-space comparison can capture — a boundary the paper partially maps through its BRIGHT results, where ReasonRank (35.7) still holds a lead over E2RANK-8B (33.4). For routine retrieval tasks — web search, document ranking, factoid QA — the generation-based reranker's latency premium appears difficult to justify given E2RANK's demonstrated performance.

Finally, the paper demonstrates that ranking-aware training improves embedding quality, a finding with implications beyond reranking. The consistent retrieval gains from Stage II training — +1.58 NDCG for 8B, +0.97 for 4B, +3.67 for 0.6B on MTEB retrieval — suggest that the contrastive learning paradigm that dominates embedding model development may be leaving performance on the table by not incorporating pairwise relative relevance signals. This challenges the field's default assumption that binary relevance labels in a contrastive framework are sufficient for optimal embedding quality, and opens the door to embedding models that are trained with richer ranking supervision even when they are intended only for retrieval.

Follow-Up Research This Work Enables

Cross-model-family replication and the role of instruction tuning. The paper evaluates exclusively on Qwen3-family models. A replication study across Llama-3, Mistral, Gemma, and other widely used embedding backbones would establish whether E2RANK's gains are specific to Qwen3's architecture, instruction-tuning recipe, or chat template format, or whether they generalize to decoder-only LLM embeddings broadly. The study should control for model scale (e.g., comparing 7B/8B models from each family), training data (using the same Stage I and Stage II datasets), and inference configuration. Key measurements: reranking NDCG on BEIR, embedding quality on MTEB, and the contribution of the chat template (ablating whether the template tokens matter or the method works with plain instruction formatting). A negative result — substantial family-dependent variation — would bound the method's generality and motivate family-specific tuning. A positive result — consistent gains across families — would establish E2RANK as a generic recipe applicable to any decoder-only embedding model. The Qwen3-specific chat tokens (<|im_start|>user, <|im_end|>) are a particular concern: the model may be using these as structural cues to activate different processing modes, and other families' format tokens may not provide equivalent signal.

Scaling the PRF mechanism: how many documents are optimal and when? Figure 2 establishes a saturation curve on two TREC DL datasets, but the optimal number of input documents likely depends on document length, query type, and initial retrieval quality. A systematic study would measure the saturation curve on each BEIR dataset (NFCorpus with very short documents, SciFact with longer scientific abstracts, Touché2020 with argumentative text) and on BRIGHT reasoning tasks, testing document counts from 5 to 100. The study would also bucket queries by initial retrieval quality (e.g., whether the correct document appears in the top-5, top-20, or not in the top-100) to test whether harder queries benefit from more PRF documents (because they need more signal to find the correct answer) or fewer (because including more low-quality documents introduces noise). A strong result would produce a practical guideline — e.g., "optimal document count = min(20, the number of documents with estimated relevance above threshold T)" — that practitioners could implement without per-dataset hyperparameter sweeps. The paper's finding that stronger retrievers produce better reranking (Appendix D, Table 21) predicts a positive correlation between initial retrieval quality and optimal document count, but this has not been measured.

Closing the reasoning gap: can embedding-based reranking match generation-based reasoning on BRIGHT if given equivalent training data? E2RANK trails ReasonRank-7B by 2.3 NDCG on BRIGHT (33.4 vs. 35.7), but ReasonRank was trained on synthetic reasoning data while E2RANK used general retrieval datasets. A controlled experiment would train E2RANK on the same synthetic reasoning data as ReasonRank, using the same two-stage pipeline (Stage I contrastive on retrieval data, Stage II multi-task on reasoning-labeled ranking data), and measure whether the gap closes, narrows, or persists. If E2RANK matches ReasonRank with equivalent data, the embedding bottleneck is not fundamental — it can be overcome by training signal — and the remaining efficiency advantage (no autoregressive generation) makes embedding-based reasoning reranking strictly preferable. If a gap persists even with equivalent data, the embedding representation is indeed a bottleneck for complex reasoning, and the field gains a clearer map of which tasks require explicit reasoning chains versus which can be handled through embedding-space comparisons. This experiment would also inform whether hybrid systems — embedding-based scoring for most documents, with generation-based fallback for the hardest cases — are worth pursuing.

Training data efficiency: how much Stage II data is necessary? The paper uses ~87K LLM-labeled ranking examples for Stage II training, but provides no ablation on data scale. A data scaling experiment — training E2RANK on subsets of the Stage II data (5K, 10K, 25K, 50K, 87K examples) — would reveal the minimum viable data budget and the shape of the data scaling curve for reranking performance. This is practically important because generating ranking labels with a large LLM (Qwen3-32B in this case) incurs non-trivial cost, and practitioners need to know whether they can achieve acceptable performance with fewer labeled examples. The experiment would also measure whether the data scaling curve differs across embedding quality (InfoNCE component) and reranking quality (RankNet component) — the paper's multi-task design may mean that less ranking data is needed because the contrastive loss provides a strong regularization toward the embedding foundation. A finding that 25K examples achieves 90% of the full-data performance would make the approach substantially more accessible.

Alternative ranking losses and their interaction with the PRF mechanism. The paper uses RankNet loss for Stage II training, but the learning-to-rank literature offers many alternatives — LambdaRank (which weights pairs by NDCG impact), ListNet (which operates on full listwise probability distributions), and various pointwise losses. A systematic comparison would train E2RANK variants with each loss function, controlling for training data and hyperparameters, and measure reranking performance on BEIR and BRIGHT. This experiment would answer two questions: whether RankNet is indeed optimal for the PRF-based reranking task (or whether listwise losses that directly optimize NDCG perform better), and whether different losses produce embedding spaces with different properties (e.g., LambdaRank might produce sharper score separation at high ranks at the expense of lower-rank discrimination). A negative result — all losses perform similarly — would support the paper's claim that the PRF mechanism, not the specific loss choice, drives the gains. A positive result — specific losses systematically outperform — would improve the method directly and provide guidance for practitioners.

Real-time difficulty estimation and adaptive PRF allocation. The paper applies a fixed strategy to all queries (top-20 documents in the listwise prompt to rerank top-100), but the PRF framework suggests that optimal allocation should be query-dependent. A follow-up would develop a lightweight query difficulty estimator — perhaps a small classifier trained on features like initial retrieval score distribution, query length, or the PRF-enriched embedding's own confidence signal — that predicts, before reranking, how many documents to include in the listwise prompt and whether to use E2RANK or fall back to a generation-based reranker for hard cases. The evaluation would measure whether adaptive allocation improves the accuracy-efficiency Pareto frontier relative to the fixed strategy, and whether the difficulty estimator's overhead (additional inference) is justified by the gains. This would directly address E2RANK's limitation of uniform strategy allocation and connect to the broader literature on compute-optimal inference strategies.

Practical Applications and Downstream Use Cases

Single-model search pipelines for latency-sensitive production systems. E2RANK enables a deployment architecture where one model handles both retrieval and reranking, reducing system complexity, memory footprint, and maintenance burden. For a web search or enterprise document retrieval system processing thousands of queries per second, the 5× latency reduction over generation-based reranking (16.93s → 3.40s per query at 8B scale on TREC COVID) directly translates to lower serving costs or higher throughput. The document embeddings can be precomputed and indexed, so online latency scales primarily with the listwise prompt encoding cost. Using E2RANK-8B, the online-only latency of 0.64 seconds per query (Table 12) is compatible with interactive search scenarios, while still delivering competitive ranking quality (55.08 NDCG on BEIR end-to-end, Table 5). For organizations currently running separate embedding retriever + cross-encoder reranker pipelines, consolidating to a single E2RANK model eliminates the need to maintain two inference services and simplifies the indexing pipeline (since document embeddings serve both stages).

Cost-effective batch reranking for RAG data preparation and evaluation. When building retrieval-augmented generation (RAG) systems, practitioners often need to rerank large volumes of retrieved passages to select the most relevant ones for inclusion in LLM prompts. Generation-based listwise rerankers are prohibitively expensive for this batched offline processing (e.g., reranking top-100 passages for 1 million queries at 16.93s per query would take ~196 GPU-days on a single A100). E2RANK's batch inference capability — encoding many listwise prompts in parallel — combined with reusable document embeddings makes this feasible: with precomputed documents, the online reranking cost per query is 0.64s for 8B, reducing the same million-query batch to ~7.4 GPU-days. The competitive accuracy on BEIR (54.35 NDCG for 8B) and, with suitable training data, on reasoning tasks (33.4 on BRIGHT) means the quality loss relative to generation-based reranking is small relative to the cost savings.

On-device or edge deployment with small E2RANK variants. The 0.6B model's strong performance — 52.09 NDCG on BEIR, outperforming RankQwen3-0.6B by +4.06 points and running in 0.63 seconds total (0.13s online-only) — makes it viable for on-device search where LLM-based reranking is infeasible due to memory and compute constraints. A mobile search application could run E2RANK-0.6B locally for both retrieval (against an on-device document index) and reranking, providing listwise-quality ranking without network latency or server costs. The model's maintenance of embedding quality on MTEB (61.25 average for 0.6B, competitive with dedicated embedding models in its size class) means the same on-device model could serve multiple text processing tasks — search, classification, clustering — reducing the total model footprint.

Data annotation and distillation pipelines using PRF-based quality estimation. E2RANK's scoring mechanism — the similarity between the PRF-enriched query embedding and each document — produces calibrated relevance scores that can serve as automatic quality estimates for distilling ranking knowledge into smaller or faster models. For example, a large E2RANK-8B could score millions of query-document pairs, and the scores (which reflect listwise comparative information) could be used as soft labels to train an even smaller student reranker or to filter training data for a retrieval model. Because the scores are continuous and discriminative (Figure 3 shows the listwise prompt sharpens score distributions), they may provide richer training signal than binary relevance judgments or single-model pointwise scores. This application leverages E2RANK's ability to encode comparative document information into individual document scores — each score implicitly reflects the document's standing relative to the other candidates used in the PRF prompt, even though the scoring is done via simple cosine similarity.