Semantic IDs for Joint Generative Search and Recommendation

🎯 Pitch

Training separate Semantic IDs for search and recommendation in a unified model breaks the other task—search-tuned IDs slash recommendation recall by 60%, while rec-tuned IDs nearly kill search. Yet a single embedding space, jointly fine-tuned with both contrastive objectives, matches task-specific performance in each domain without doubling the vocabulary.

1. Executive Summary

This paper studies how to construct Semantic IDs — discrete token sequences derived from item embeddings — that perform well for both search and recommendation when used in a unified generative model, systematically comparing task-specific and cross-task construction strategies on the MovieLens25M dataset with a Flan-T5-base generative model. Task-specific Semantic IDs optimized for a single task excel in isolation but sacrifice performance on the other task (search-tuned IDs reduce recommendation effectiveness by 60%; recommendation-tuned IDs yield near-zero search recall), while a cross-task approach — fine-tuning a bi-encoder jointly on both search and recommendation signals to produce Multi-task embeddings (a shared encoder trained with the sum of contrastive losses from query–item and co-occurring item pairs) — provides an effective trade-off, achieving the highest balanced performance (search R@30 = 0.046, recommendation R@30 = 0.049) without inflating the token budget. The paper establishes that a shared embedding space can reconcile the fundamental tension between search and recommendation fidelity, challenging the conventional wisdom that optimal generative performance requires per-task ID spaces — but only when both task objectives inform the embedding model before discretization, since post-hoc fusion of separately trained embeddings (Fusedconcat, FusedSVD) or token-level separation (Separate, Prefix-share) underperforms the joint training approach.

2. Context and Motivation

The Core Problem: Representation Collision in Unified Generative Models

The paper addresses a problem that emerges at the intersection of two converging trends in information retrieval and recommender systems. The first trend is the use of generative models — specifically encoder-decoder transformers like T5 — as a unified architecture that can serve both search and recommendation queries from a single model. The second trend is the use of Semantic IDs: discrete token sequences that represent items in these generative models, constructed by quantizing item embeddings into codebook indices.

The collision occurs here: when a single generative model must perform both search and recommendation, what item representations should it use? Prior work on Semantic IDs for generative retrieval has consistently optimized item embeddings for a single task — search-oriented embeddings for information retrieval models, collaborative-filtering embeddings for recommendation models. But a joint search-and-recommendation (Joint S&R) model cannot use two different representations for the same item simultaneously without a strategy for reconciling them. The paper frames this as a representation tension: embeddings fine-tuned for one task degrade performance on the other, and no prior work has systematically studied how to construct Semantic IDs that work well for both tasks in a unified generative setting.

Why This Problem Matters Now

The problem's importance has both practical engineering dimensions and deeper theoretical implications for how item representations are learned in multi-task systems.

Practical importance: consolidation of infrastructure. The motivating context (Section 2, echoed in the Introduction) is that search and recommendation systems have historically been built as "separated silos" — different models, different data pipelines, different embedding spaces, different serving stacks. A unified model that handles both tasks would reduce engineering overhead, simplify maintenance, and potentially enable knowledge transfer between tasks (a point the paper cites from Penha et al., 2024). However, that unification is only valuable if the unified model performs competitively on both tasks. If the item representations baked into the model are fundamentally biased toward one task, the unification breaks down: you get a model that searches well but recommends poorly, or vice versa. The paper's core practical question — how to build Semantic IDs that don't sacrifice either task — is therefore a prerequisite for realizing the engineering benefits of unification.

Theoretical importance: representation learning under competing objectives. The tension between search and recommendation embeddings reflects a deeper problem in multi-task representation learning. Search relies on content-based item representations: the embedding should capture semantic similarity between a query and an item's metadata (title, description, genres). Recommendation relies on collaborative filtering signals: the embedding should capture patterns of co-interaction among users, where items are similar if they are consumed by overlapping users. These two sources of similarity do not necessarily align. A documentary and a blockbuster action film may share almost no content overlap but be co-watched by the same audience. Conversely, two documentaries on the same topic may have nearly identical metadata but very different audience profiles. Embeddings optimized for one signal will systematically misrepresent the other.

This tension is not niche to movie recommendation — it appears whenever systems must reconcile content-based and collaborative similarity (e.g., e-commerce with product descriptions plus purchase histories, music streaming with audio features plus listening patterns, academic search with paper text plus citation graphs). The paper's investigation of how to construct joint Semantic IDs is therefore a case study in a more general question: can a single embedding space capture multiple, partially conflicting notions of item similarity? The answer matters for any multi-task system that needs to represent items for diverse downstream objectives.

Prior Approaches and Where They Fall Short

To understand where the paper positions itself, it is essential to survey the prior approaches to item representation in generative models and identify their specific limitations in a joint setting.

Approach 1: Content-based Semantic IDs (DSI, TIGER). Early work on generative retrieval, particularly DSI (Tay et al., 2022) and TIGER (Rajput et al., 2023), constructed Semantic IDs by quantizing text-based item embeddings derived from pre-trained language models. These embeddings capture the semantic content of items — their titles, descriptions, and metadata — and are not fine-tuned for any particular retrieval or recommendation task. The resulting Semantic IDs are content-grounded and generalize to unseen items (cold start), which is a significant practical advantage.

However, as Table 1 and Table 2 show, content-based embeddings underperform substantially on both search and recommendation compared to task-tuned alternatives. A content-based bi-encoder trained with no task-specific fine-tuning achieves search R@30 of only 0.013 (Table 1, row 1), versus 0.072 for search-tuned embeddings. On recommendation, it achieves 0.023, versus 0.062 for recommendation-tuned embeddings. The gap is large: content alone is insufficient to capture either retrieval relevance or collaborative filtering patterns with competitive accuracy. This motivates the need for fine-tuning — but fine-tuning necessarily introduces a task bias.

Approach 2: Search-tuned Semantic IDs (RIPOR). Zeng et al. (2024) proposed RIPOR, which fine-tunes a bi-encoder on query–item pairs with in-batch contrastive loss to produce embeddings specifically optimized for search relevance. The resulting Semantic IDs significantly improve generative retrieval performance. This paper reproduces that pipeline (Section 3.1, "Search-based"): train a bi-encoder on search data ( $\mathcal{D}_S$ ) composed of query–relevant-item pairs, then quantize the resulting embeddings $\mathbf{v}_{\text{search}}$ to produce Semantic IDs.

The problem, documented in Table 1 (row 2), is that search-tuned IDs degrade recommendation performance by approximately 60% compared to recommendation-tuned IDs (recommendation R@30 drops from 0.062 to 0.026). The reason is that the search bi-encoder produces embeddings that reflect query–document relevance — a notion of similarity driven by content overlap between natural language queries and item metadata. This embedding space does not capture collaborative filtering signals (who interacts with what), which are critical for recommendation. Items that are similar in query-space may be distant in interaction-space, and vice versa.

A subtle but important point in the paper's results is how this plays out across popularity levels. Table 2 shows that on "Head" items (the top 1% most popular), search-tuned IDs achieve recommendation R@30 of 0.090 — not terrible, because popular items tend to appear in many interactions and their metadata may correlate with their audience. On "Torso" items (the remaining 99%), search-tuned IDs achieve recommendation R@30 of 0.070, which is still reasonable — the authors note this demonstrates that "for less popular items leaning more on content is effective." For less popular items with sparse interaction data, content-based signals become more useful, and search embeddings partially substitute for collaborative filtering. But the overall recommendation performance is substantially below what a dedicated collaborative-filtering embedding can achieve on popular items (0.170 for recommendation-tuned on Head items).

Approach 3: Recommendation-tuned Semantic IDs (TokenRec). Qu et al. (2024) proposed TokenRec, which trains a collaborative-filtering model (Efficient Neural Matrix Factorization, or ENMF; Chen et al., 2020) on user–item interaction data to produce embeddings $\mathbf{v}_{\text{rec}}$ that capture collaborative filtering patterns. These embeddings are then quantized to produce Semantic IDs for a generative recommendation model.

The mirror-image problem appears: recommendation-tuned IDs achieve strong recommendation performance but catastrophically fail at search, with search R@30 of only 0.004 (Table 1, row 3) — essentially zero. The collaborative-filtering embeddings carry no information about the item's content or how it relates to natural language queries. A user typing "thought-provoking sci-fi about memory" cannot match that query to collaborative-filtering embeddings that only encode patterns of who watched what together. This is a harder failure mode than the search→recommendation degradation because search requires content grounding that recommendation-only embeddings fundamentally lack.

Approach 4: Separate task tokens (stated but not from prior work). A natural intuition for handling competing objectives is to give each task its own representation space: generate separate Semantic IDs from search embeddings and recommendation embeddings, prefix them with task tags, and have the generative model learn to produce the correct ID sequence depending on the task. This is the Separate approach the paper introduces and evaluates in Section 3.2.

The results (Table 2, "Separate") show that this approach underperforms: search R@30 = 0.028 and recommendation R@30 = 0.032, both substantially below their task-specific counterparts (0.072 and 0.062, respectively). Why does separate representation space fail? The paper provides an explanation that draws on Penha et al. (2024): "the knowledge learned from one task cannot be used for the task-specific tokens of the other task, negating the regularization effect in item representations." In other words, when the generative model sees the search tokens for a given item, it learns representations that could help recommendation — but the model has no mechanism to transfer that knowledge because recommendation operates on a completely separate vocabulary of tokens. The two sets of tokens are disjoint, so the model cannot exploit cross-task regularities. An item that is retrieved for many search queries might also be popular in recommendations, but the model cannot leverage this correlation because the search tokens and recommendation tokens are different vocabulary entries with no shared parameters (beyond what the transformer backbone learns indirectly).

Additionally, the Separate approach doubles the number of new tokens added to the vocabulary (1024 vs. 512 in the standard two-codebook setup), increasing model size and training complexity.

Approach 5: Prefix-shared IDs (Shi et al., 2025). A concurrent approach proposed by Shi et al. (2025) addresses the representation collision by allocating three codebooks: a shared codebook plus two task-specific ones. A single encoder processes the concatenated embeddings $[\mathbf{v}_{\text{search}}; \mathbf{v}_{\text{rec}}]$ , and two decoders learn reconstructions for the shared and task-specific portions. The final Semantic ID concatenates shared tokens followed by task-specific tokens.

The paper implements this as Prefix-share and reports that it underperforms other cross-task approaches (search R@30 = 0.007, recommendation R@30 = 0.021) — in fact performing worse than even the task-specific baselines for both tasks. The authors attribute this to "the underlying quantization approach not performing well here" (referring to the ablation in Table 3 showing that the auto-encoder-based quantization method, which Prefix-share requires for its decoder architecture, underperforms simpler RQ-KMeans). This is a revealing negative result: it suggests that the architectural complexity of learning separate codebook reconstructions may not be worth the quantization performance penalty, at least with current auto-encoder quantization methods.

The common thread across these prior approaches is that they all optimize item embeddings for a single task (or treat task-specific embeddings independently) and then attempt to use those embeddings — or their concatenation/separation — in a joint model. None explicitly trains an embedding model on both supervision signals simultaneously, so none resolves the underlying tension in the embedding space itself. This is the gap that the paper's Multi-task approach directly addresses.

How This Paper Positions Itself

The paper positions itself as addressing a design choice in unified generative retrieval systems: how to construct item embeddings (and consequently Semantic IDs) when the system must perform both search and recommendation. This is not a novel architecture proposal or a new training algorithm — it is a systematic empirical study of a decision that every joint generative model builder must make, but that prior work has either ignored (by focusing on single-task systems) or addressed with ad-hoc solutions (concatenation, separate tokens) without thorough comparison.

The paper's central research question is stated directly in the Introduction:

"Can we create Semantic IDs that perform well for both search and recommendation in a joint generative model?"

This question is operationalized through a comparison of five cross-task strategies (Separate, Prefix-share, Fusedconcat, FusedSVD, Multi-task) against two task-specific baselines (Search-based, Recommendation-based), all evaluated under identical conditions: the same generative model architecture (Flan-T5-base), the same dataset (MovieLens25M), the same tokenization procedure (RQ-KMeans with two codebooks of size 256), and the same training regime (3 epochs, jointly trained on search and recommendation prompts).

The paper does not claim to have "solved" the representation tension. Its contributions are empirical and diagnostic:

Quantifying the tension. Table 2 provides a clear, numbered account of the trade-off: search-tuned IDs give search R@30 of 0.072 but recommendation R@30 of 0.026; recommendation-tuned IDs flip this to 0.004 and 0.062. This establishes a concrete upper and lower bound for each task.
Showing that post-hoc fusion underperforms joint training. Fusedconcat and FusedSVD — which combine separately trained search and recommendation embeddings — do not reach the performance of Multi-task, which trains a single encoder on both objectives simultaneously. Fusedconcat achieves search R@30 of 0.048 and recommendation R@30 of 0.018; Multi-task achieves 0.046 and 0.049. The recommendation gap is substantial: Multi-task more than doubles Fusedconcat's recommendation performance while sacrificing only 0.002 in search. This is the paper's key finding: joint training in the embedding space, not post-hoc composition, is what enables the trade-off.
Demonstrating that token separation fails. Separate and Prefix-share, which give each task its own ID tokens, underperform approaches that create a single shared Semantic ID from combined embeddings. This suggests that shared item representations — where the same tokens represent the same item regardless of task — are beneficial, likely because they force the generative model to learn task-agnostic properties of items and because they enable cross-task knowledge transfer during training.
Positioning the Multi-task approach as a Pareto-efficient compromise. Figure 1 visualizes the trade-off frontier: Search-based IDs occupy the far-right (high search, low recommendation), Recommendation-based IDs the far-up (high recommendation, low search), and Multi-task sits in the upper-right region neither dominates. The paper explicitly frames this as a "compelling compromise" that achieves strong performance on both tasks without increasing token budget or model complexity.

The paper also positions itself relative to concurrent work (Shi et al., 2025, which proposes Prefix-share) and relative to its own prior work (Penha et al., 2024, which studied whether search and recommendation help each other in generative models but used content-based embeddings, not task-tuned ones). The key advance over Penha et al. is the explicit investigation of how the embedding space — and specifically its fine-tuning objective — determines the effectiveness of the resulting Semantic IDs in a joint model. The earlier work established that joint training of the generative model helps; this work establishes that the embedding space used to construct the IDs is equally critical.

A Note on the Dataset and Generalizability Expectations

The paper builds a search-recommendation dataset from MovieLens25M by generating 20 synthetic queries per movie using Gemini-2.0-flash (10 for training, 10 for evaluation). This design choice is worth highlighting because it shapes the interpretation of results.

MovieLens25M is a standard recommendation benchmark with user–item interaction data, but it lacks real search logs. The authors must therefore simulate search behavior, and they make a specific choice: generate a uniform number of queries per item (exactly 10 train queries and 10 test queries per movie). This means the search data has no popularity bias — every item, regardless of how many users interacted with it, has exactly the same number of queries. This is an important caveat the authors explicitly acknowledge:

"Given that we do not know the true distribution of popularity in search, i.e. there are no real user logs for the MovieLens data, we decided on using the uniform distribution. This means that the search popularity distribution is quite different from the recommendation distribution, and thus we might expect results to be more favorable in real-life distributions with some similarity between popularity distributions."

In real-world search–recommendation systems, popular items in recommendation (head movies, popular products) are often also frequent targets of search queries. The correlation between search popularity and recommendation popularity provides an additional signal that could make the Multi-task approach even more effective in production settings, since the two supervision signals would reinforce rather than contradict each other for popular items. The uniform query distribution in this dataset likely represents a harder setting for joint learning than real deployments, because there is no natural overlap between "items people search for" and "items people interact with." Results on this dataset may therefore underestimate the practical benefits of joint embedding training.

3. Technical Approach

3.1 Reader Orientation

This paper is an empirical comparison study that systematically evaluates different strategies for constructing Semantic IDs — discrete token sequences representing items — when those IDs must serve both search and recommendation in a single generative transformer model. The core problem is that embeddings optimized for one task produce Semantic IDs that fail on the other task, and the paper's solution is to show that jointly training a bi-encoder on both search and recommendation supervision signals before quantization produces embeddings (and consequently Semantic IDs) that perform competitively on both tasks without requiring separate ID spaces or post-hoc embedding fusion.

3.2 Big-Picture Architecture (Diagram in Words)

The system has five major components arranged in a sequential pipeline:

Item Metadata and Interaction Data — the raw inputs: for each item, structured text fields (title, year, description, genres, tags, genome tags) plus user–item interaction logs. For search, the dataset includes 10 training queries and 10 test queries per item, synthetically generated by Gemini-2.0-flash. These two data streams provide the supervision for the two tasks.
Embedding Models — neural networks that map each item to a dense vector. Different configurations are tested: a content-based model (all-mpnet-base-v2 from Sentence Transformers), a search-tuned bi-encoder (same architecture, fine-tuned on query–item pairs), a recommendation-tuned collaborative filtering model (Efficient Neural Matrix Factorization, ENMF), and a Multi-task bi-encoder trained on both signals simultaneously. Each embedding model produces an item vector $\mathbf{v} \in \mathbb{R}^d$ .
Embedding Fusion or Combination — for cross-task approaches, this component takes embeddings from multiple sources (e.g., search-tuned and recommendation-tuned) and combines them into a single vector. Methods include concatenation (Fusedconcat), dimensionality-equalized summation (FusedSVD), or — in the Multi-task case — no fusion is needed because the embedding model itself is trained on both objectives. For token-separated approaches (Separate, Prefix-share), this component instead allocates separate token spaces per task.
Quantization (ID Tokenization) — a quantization algorithm (primarily RQ-KMeans, unless otherwise ablated) maps each combined item embedding to a discrete sequence of tokens drawn from learned codebooks. For the standard configuration, two codebooks of size 256 are used, producing 512 tokens total. The quantization output is the item's Semantic ID: a tuple of codebook indices, e.g., $\langle c_1, c_2 \rangle$ where each $c_i \in \{1, \ldots, 256\}$ .
Joint Generative Model — a Flan-T5-base encoder-decoder transformer, trained jointly on both search and recommendation prompts for 3 epochs. The model receives task-tagged prompts (e.g., "Search: thought-provoking sci-fi about memory" or "Recommend: user interacted with item_1, item_2, ...") and is trained to autoregressively generate the appropriate Semantic ID tokens for relevant items. At inference, diversified beam search (beam size 60, diversity penalty 0.25, 30 groups) decodes candidate item IDs.

Information flows sequentially: raw item data → embedding model produces item vectors → (optionally) vectors are fused/combined → quantization produces discrete Semantic IDs → generative model is trained to map search/recommendation prompts to these IDs. The key design choice the paper investigates is what happens at the embedding model and fusion stages: which supervision signals shape the item vectors before they are quantized into IDs.

3.3 Roadmap for the Deep Dive

First, the formal generative retrieval setup and how Semantic IDs map items to discrete tokens, since this is the shared substrate that all construction strategies operate on.
Second, the embedding models — the content-based baseline, the search-tuned bi-encoder, the recommendation-tuned ENMF, and the Multi-task bi-encoder — because the embedding model's training objective is the primary variable being manipulated. Each model's architecture, loss function, and training procedure determines what kind of similarity its vectors capture.
Third, the five cross-task Semantic ID construction strategies (Separate, Prefix-share, Fusedconcat, FusedSVD, Multi-task), because these are the paper's core contribution: they represent different answers to the question "given embeddings from potentially different sources, how do we produce a single Semantic ID for each item?"
Fourth, the quantization (tokenization) procedure — RQ-KMeans and its alternatives — because the discretization step introduces its own trade-offs (information loss, reconstruction quality, token budget) that interact with the embedding quality, and the paper ablates different quantization methods to ensure the findings are not artifacts of a particular tokenizer.
Fifth, the joint generative model training and inference — prompt formats, training hyperparameters, and the diversified beam search decoding strategy — because these are held constant across all embedding construction strategies and constitute the evaluation substrate.

3.4 Detailed, Sentence-Based Technical Breakdown

This is an empirical ablation study whose core idea is that the embedding space used to construct Semantic IDs is the decisive factor in joint search-and-recommendation performance, and that training a single encoder on both supervision signals (Multi-task) outperforms post-hoc fusion of separately trained embeddings or token-level separation of task-specific IDs.

Generative Retrieval with Semantic IDs

The paper operates within the generative retrieval paradigm, where items are not ranked by computing similarity scores between a query embedding and item embeddings (as in dual-encoder retrieval), but instead are directly generated as token sequences by an autoregressive language model.

Item representation as tokens. For a generative model to output an item, that item must be representable as a sequence of tokens from the model's vocabulary. Unlike natural language tokens (words, subwords), items do not have a natural tokenization. The paper uses Semantic IDs: each item is assigned a tuple of discrete codebook indices obtained by quantizing an item embedding vector.

Concretely, given an item embedding $\mathbf{v} \in \mathbb{R}^d$ and a set of $K$ codebooks each of size $V$ (typically $K=2$ , $V=256$ ), the quantization process maps:

$\mathbf{v} \mapsto \langle c_1, c_2, \ldots, c_K \rangle$

where $c_i \in \{1, \ldots, V\}$ is the index of the closest centroid in the $i$ -th codebook. The resulting Semantic ID is a sequence of $K$ tokens, each drawn from a codebook-specific vocabulary of size $V$ . With two codebooks of size 256, this yields 512 new tokens added to the generative model's vocabulary (256 per codebook).

Why Semantic IDs. The paper motivates Semantic IDs over alternative item representations (Section 1, citing Rajput et al., 2023 and Tay et al., 2022) with two key properties:

Generalization through token sharing. Items with similar embeddings share tokens in their Semantic IDs (because they fall into the same Voronoi cells of the codebooks), enabling the generative model to leverage similarity structure. An item never seen during training can still be represented if its embedding lands in existing codebook regions, enabling cold-start handling without retraining.
Decoupling from raw item text. Alternative approaches that use item titles or descriptions as token sequences (De Cao et al., 2020) are less token-efficient and tie the generative model to surface-form text rather than learned semantic structure. Semantic IDs compress the item representation into a small, fixed number of tokens (typically 2–4) regardless of item complexity.

The generative task formulation. During training, the joint generative model receives prompts of two types:

Search prompt: "Search: {query}" → target output: Semantic ID tokens of relevant items
Recommendation prompt: "Recommend: {user interaction history}" → target output: Semantic ID tokens of next-item predictions

The model is trained with standard autoregressive cross-entropy loss: given the prompt as input to the encoder, the decoder is trained to predict the next token of the target Semantic ID at each position. At inference time, the same prompts are provided, and the decoder generates candidate Semantic ID sequences via beam search.

What determines Semantic ID quality. Since the quantization step (RQ-KMeans) is a deterministic function of the input embedding, the quality of Semantic IDs is entirely determined by the quality of the embedding vectors they are constructed from. This is the central insight that motivates the paper's investigation: changing the embedding model changes everything downstream. If the embedding space conflates items that are similar for search but dissimilar for recommendation, the resulting Semantic IDs will encode that conflation, and the generative model — no matter how well trained — cannot recover task-appropriate similarity from misaligned tokens.

Token budget considerations. Each codebook of size $V$ adds $V$ new tokens to the generative model's vocabulary. These tokens have their own learned embeddings in the model's input and output layers, increasing parameter count. The standard configuration adds 512 tokens. Some cross-task strategies (Separate) double this to 1024 tokens by allocating separate codebooks per task. The paper considers token budget an important practical constraint: larger vocabularies increase model size and training time, and more tokens per item increase sequence length and decoding cost. A key strength claimed for the Multi-task approach is that it achieves balanced performance without inflating the token budget.

Embedding Models

The embedding model is the component that maps each item to a dense vector $\mathbf{v} \in \mathbb{R}^d$ . The paper constructs and compares four distinct embedding sources:

1. Content-based embedding. This baseline uses the pre-trained all-mpnet-base-v2 model from Sentence Transformers (Reimers and Gurevych, 2019), a 768-dimensional bi-encoder based on MPNet (a BERT variant with permuted language modeling pre-training). The model receives as input the concatenated item metadata (title, year, description, genres, tags, and genome tags from Vig et al., 2012) and produces a fixed-size embedding via mean pooling over the final hidden states.

This model is not fine-tuned on any task-specific data. It captures general semantic similarity as learned from large-scale natural language pre-training — items with similar textual descriptions receive similar embeddings. The paper uses this as a "content-only" lower bound to quantify how much task-specific fine-tuning improves performance.

2. Search-tuned bi-encoder embedding. The search-tuned embedding starts from the same pre-trained all-mpnet-base-v2 checkpoint and is further fine-tuned on the search dataset $\mathcal{D}_S$ .

Training data. The search dataset $\mathcal{D}_S$ consists of query–item pairs where the queries are the 10 training queries per item generated by Gemini-2.0-flash (described in Section 4) and the items are the corresponding movies. Each movie's document representation is the concatenated metadata (title, year, description, genres, tags, genome tags), matching the content-based model's input.

Loss function. The model is trained with the MultipleNegativesRankingLoss from Sentence Transformers, which is an in-batch contrastive loss. For a batch of $B$ query–item pairs $\{(q_i, d_i^+)\}_{i=1}^B$ , the loss for query $q_i$ is:

$\mathcal{L}_i = -\log \frac{\exp(\text{sim}(q_i, d_i^+) / \tau)}{\sum_{j=1}^B \exp(\text{sim}(q_i, d_j^+) / \tau)}$

where $\text{sim}(\cdot, \cdot)$ is cosine similarity and $\tau$ is a temperature parameter (implicitly 1.0 in the MultipleNegativesRankingLoss default, which uses cosine similarity without explicit temperature scaling). The key mechanism is in-batch negatives: for each query, all other items in the same batch serve as negative examples, since they are known not to be relevant to that query (they are paired with different queries). This makes the loss computationally efficient — no explicit negative sampling is needed — and scales the effective number of negatives with batch size.

Training hyperparameters (Section 4, "For search"). The bi-encoder is fine-tuned for 5 epochs with batch size 512, learning rate $2\times10^{-5}$ , using the Adam optimizer. The resulting embeddings $\mathbf{v}_{\text{search}}$ are 768-dimensional (matching the MPNet output dimension).

What it captures. After fine-tuning, $\mathbf{v}_{\text{search}}$ represents each item in a space optimized for query–document relevance: items that are relevant to the same queries (or semantically similar queries) cluster together. This captures content-based similarity (since queries describe what users search for and items are represented by their metadata), but it is specifically shaped by the supervision signal of which items are relevant to which queries.

3. Recommendation-tuned collaborative filtering embedding. The recommendation-tuned embedding is produced by an Efficient Neural Matrix Factorization (ENMF) model (Chen et al., 2020), trained on the recommendation dataset $\mathcal{D}_R$ consisting of user–item interaction pairs.

Model architecture. ENMF is a matrix factorization model that represents each user and each item with a learned embedding of dimension 256. The predicted affinity between user $u$ and item $i$ is the dot product of their embeddings: $\hat{r}_{ui} = \mathbf{u}_u^\top \mathbf{v}_i^{\text{rec}}$ . ENMF's key property, as its name suggests, is efficient training without negative sampling — instead of sampling negative items for each user–item pair (as in Bayesian Personalized Ranking or standard matrix factorization), ENMF reformulates the loss to operate over all items simultaneously using a weighted regression objective, making it computationally tractable for large item catalogs.

Training hyperparameters (Section 4, "For recommendation"). The ENMF model is trained for 30 epochs with batch size 512, embedding dimension 256, learning rate 0.001, using the Adam optimizer. The implementation uses RecBole (Zhao et al., 2022).

What it captures. The item embeddings $\mathbf{v}_{\text{rec}}$ from ENMF capture collaborative filtering signals: items are close in the embedding space if they are consumed by overlapping sets of users. This is fundamentally different from content-based similarity — two items with completely different metadata may receive similar embeddings if they share an audience. The embedding dimension (256) is smaller than the bi-encoder (768), which matters for the fusion strategies discussed below.

4. Multi-task bi-encoder embedding. The Multi-task embedding model is trained on both $\mathcal{D}_S$ (query–item pairs) and $\mathcal{D}_R$ (co-occurring item pairs) simultaneously using a shared encoder with two contrastive losses.

Model architecture. The model starts from the same pre-trained all-mpnet-base-v2 checkpoint and uses the same encoder architecture as the search-tuned bi-encoder. The key difference is the training objective: a sum of two contrastive losses.

Training data for recommendation. The recommendation data $\mathcal{D}_R$ consists of co-occurring item pairs — pairs of items that appear together in a user's interaction history. This transforms the collaborative filtering signal (user–item interactions) into a format compatible with bi-encoder training (item–item pairs). Specifically: if a user interacted with items $i_1, i_2, i_3$ in sequence, all pairwise combinations $(i_a, i_b)$ for $a \neq b$ can be treated as positive item–item pairs. The bi-encoder takes the metadata of one item as input and is trained to predict the metadata of a co-occurring item as relevant.

Loss function. The total training loss is the sum:

$\mathcal{L}_{\text{multi-task}} = \mathcal{L}_{\text{search}} + \mathcal{L}_{\text{rec}}$

where $\mathcal{L}_{\text{search}}$ is the MultipleNegativesRankingLoss over query–item pairs (identical to the search-tuned model's loss) and $\mathcal{L}_{\text{rec}}$ is the same contrastive loss applied to co-occurring item–item pairs. In the recommendation loss, for a batch of item–item pairs $\{(i_a, i_b^+)\}$ , item $i_a$ is encoded through the bi-encoder (using its metadata) to produce a query-like representation, and item $i_b^+$ is encoded (also through the same bi-encoder, with shared weights) to produce a document-like representation. The in-batch contrastive loss then pulls the embeddings of co-occurring items together while pushing apart embeddings of items from different pairs in the batch.

Training hyperparameters (Section 4, "Generative model"). The multi-task bi-encoder is fine-tuned for 5 epochs with batch size 512, learning rate $2\times10^{-5}$ , using Adam. These hyperparameters match the search-tuned fine-tuning, making the comparison between search-only and multi-task training a clean ablation: any difference in downstream performance is attributable only to the presence or absence of the recommendation loss term.

What it captures. The resulting embeddings $\mathbf{v}_{\text{mt}}$ are optimized to simultaneously satisfy two constraints:

Items relevant to similar search queries should be close (from $\mathcal{L}_{\text{search}}$ )
Items that co-occur in user interaction histories should be close (from $\mathcal{L}_{\text{rec}}$ )

These two objectives can conflict — as discussed in Section 2, search relevance and collaborative similarity are not always aligned — and the shared encoder must find a compromise that balances the gradients from both loss terms. The paper's key empirical claim is that this compromise produces embeddings that work well for both downstream tasks, better than any post-hoc combination of separately trained single-task embeddings.

Why item–item pairs for the recommendation signal. An important design choice is how to inject collaborative filtering information into a bi-encoder that takes item metadata as input. The item's metadata (title, description) does not contain collaborative filtering information — it only describes the item's content. Training with co-occurring item pairs creates a learning signal where the encoder must produce similar embeddings for items that have similar metadata (to satisfy the search loss) and for items that appear together in user histories (to satisfy the recommendation loss). This forces the encoder to learn representations that encode both content similarity and collaborative similarity — even though the input is only metadata. The mechanism is that items which co-occur frequently with the same set of other items will receive similar gradients and thus similar embeddings, regardless of whether their metadata is similar.

Cross-Task Semantic ID Construction Strategies

Given embeddings from one or more sources, the next step is to produce a single Semantic ID per item that the generative model will learn to output for both search and recommendation prompts. The paper introduces and compares five strategies for this step. They fall into two conceptual categories: token-level separation (Separate, Prefix-share) and embedding-level combination (Fusedconcat, FusedSVD, Multi-task).

Task-Specific Baselines (Not Cross-Task, But Provided for Reference)

Search-based. The search-tuned bi-encoder produces $\mathbf{v}_{\text{search}} \in \mathbb{R}^{768}$ . This vector is directly quantized using RQ-KMeans with two codebooks of size 256, producing a Semantic ID $\langle c_1, c_2 \rangle$ . The same ID is used for both search and recommendation in the generative model. This is a task-specific approach because the embedding only encodes search relevance.

Recommendation-based. The ENMF model produces $\mathbf{v}_{\text{rec}} \in \mathbb{R}^{256}$ . This vector is quantized identically to produce a Semantic ID. This is a task-specific approach because the embedding only encodes collaborative filtering patterns.

These two baselines establish the single-task upper bounds for each task and the cross-task failure modes: search-tuned IDs give strong search but weak recommendation; recommendation-tuned IDs give the reverse.

Token-separated IDs (Separate)

Strategy. For each item, construct two separate Semantic IDs: one from $\mathbf{v}_{\text{search}}$ (Search-based approach) and one from $\mathbf{v}_{\text{rec}}$ (Recommendation-based approach). Concatenate them with task tags to form the item's full Semantic ID:

$\text{ID}_{\text{sep}} = \langle \text{SEARCH:ID}_{\text{search}}, \text{REC:ID}_{\text{rec}} \rangle$

This is a sequence of tokens where the first tokens encode which task the ID is for, followed by the search-specific codebook tokens, followed by the recommendation-specific codebook tokens.

How the generative model uses it. During training, search prompts are paired with targets that output only the search-specific portion (after the SEARCH tag), and recommendation prompts output only the recommendation-specific portion (after the REC tag). The codebooks for the two tasks are completely disjoint: the search tokens are a separate vocabulary of 512 tokens, and the recommendation tokens are a different vocabulary of 512 tokens, adding 1024 total new tokens to the generative model's vocabulary.

Design rationale. The intuition is that giving each task its own representation space avoids the representation collision entirely. The search task gets embeddings optimized for search; the recommendation task gets embeddings optimized for recommendation. The generative model learns to route between these two spaces based on the task prompt.

Why it underperforms (Section 5). The paper reports that Separate achieves search R@30 of 0.028 and recommendation R@30 of 0.032 — both substantially below the single-task baselines (0.072 and 0.062). The authors attribute this to a missing cross-task regularization effect (citing Penha et al., 2024): "the knowledge learned from one task cannot be used for the task-specific tokens of the other task." During joint training, the generative model's encoder processes both search and recommendation prompts in the same batch. When it encodes an item for search, it builds internal representations that capture properties of that item (its popularity, its genre patterns, its audience). Those representations could, in principle, help the recommendation task — knowing that a movie is frequently searched for "thought-provoking sci-fi" might help predict it as a recommendation for users who enjoy cerebral science fiction. But because the output tokens for search and recommendation are completely disjoint vocabulary entries, the decoder's learned token embeddings for search IDs and recommendation IDs do not share parameters. The only pathway for cross-task transfer is through the shared transformer backbone parameters, which is indirect and weak compared to explicit representation sharing through common tokens.

Additionally, doubling the token budget increases model size and makes the generative model's learning problem harder: it must learn embeddings for 1024 new tokens with limited training data (3 epochs on MovieLens25M).

Strategy. This approach, adapted from concurrent work by Shi et al. (2025), allocates three codebooks: one shared codebook (SHARED) of size 256, one search-specific codebook of size 256, and one recommendation-specific codebook of size 256, totaling 768 new tokens. The embeddings from both tasks are processed through an auto-encoder quantization architecture:

Encoder. A single encoder network takes as input the concatenated embeddings $[\mathbf{v}_{\text{search}}; \mathbf{v}_{\text{rec}}] \in \mathbb{R}^{1024}$ (768 + 256 dimensions) and produces a latent representation.
Quantization and reconstruction. Two decoder networks are trained: one learns to reconstruct $\mathbf{v}_{\text{search}}$ from the latent representation, and the other learns to reconstruct $\mathbf{v}_{\text{rec}}$ . During training, the quantization step assigns the latent representation to the nearest centroids in the three codebooks. The shared codebook captures information common to both tasks; the task-specific codebooks capture task-specific residuals.
Final Semantic ID. The resulting ID for an item is the concatenation of shared tokens followed by task-specific tokens for the target task:

$\text{ID}_{\text{prefix-share}} = \langle \text{SHARED}_1, \ldots, \text{SHARED}_k, \text{TASK}_1, \ldots, \text{TASK}_k \rangle$

During search, the task-specific portion uses the search codebook; during recommendation, it uses the recommendation codebook. The shared tokens are the same for both tasks.

Design rationale. The idea is that some information about an item is task-agnostic (e.g., its general topic, its quality, its era) and can be shared via the common codebook, while task-specific information (query-relevance patterns vs. collaborative patterns) is captured in the task-specific codebooks. This aims to reduce the token budget compared to Separate (768 tokens vs. 1024) while still allowing task-specific representations.

Why it underperforms (Section 5). The paper reports that Prefix-share achieves search R@30 of 0.007 and recommendation R@30 of 0.021 — worse than both task-specific baselines and worse than all other cross-task approaches. The authors attribute this to the quantization method: Prefix-share requires an auto-encoder-based quantization architecture (to learn the shared and task-specific reconstructions), and the specific auto-encoder quantization methods available (ResidualLFQ, RQ-VAE; ablated in Table 3) perform substantially worse than the simpler RQ-KMeans approach used by all other methods.

This is an important negative finding: it suggests that the architectural complexity of learning separate codebook reconstructions introduces a quantization quality penalty that outweighs any benefit from shared-vs-specific token allocation. In other words, the degradation from worse tokenization swamps the theoretical advantage of sharing some tokens across tasks. The paper positions this as evidence that simpler is better for Semantic ID construction in the current state of quantization technology.

Fusedconcat IDs

Strategy. This approach combines the two embeddings before quantization by concatenating them:

$\mathbf{v}_{\text{concat}} = [\mathbf{v}_{\text{search}}; \mathbf{v}_{\text{rec}}]$

where both vectors are first $\ell_2$ -normalized. $\mathbf{v}_{\text{search}}$ is 768-dimensional; $\mathbf{v}_{\text{rec}}$ is 256-dimensional; $\mathbf{v}_{\text{concat}}$ is therefore 1024-dimensional. This combined vector is then quantized using RQ-KMeans (two codebooks of size 256) to produce a single Semantic ID $\langle c_1, c_2 \rangle$ that is shared across both tasks.

Design rationale. Concatenation preserves all information from both embedding spaces — no dimensionality reduction, no information loss from the embeddings themselves (only from the subsequent quantization). The $\ell_2$ -normalization ensures both vectors contribute on a comparable scale to the distance computations in the quantization step, since RQ-KMeans uses Euclidean distance to assign points to centroids.

Why it shows a search bias (Section 5). The results show Fusedconcat achieves search R@30 of 0.048 and recommendation R@30 of 0.018. This is search-heavy — recommendation performance is barely above the search-tuned baseline (0.026) and far below the recommendation-tuned baseline (0.062). The paper explains this by noting the dimensionality imbalance: "the embedding space with larger dimensionality might become more represented." The search embedding has three times the dimensionality of the recommendation embedding (768 vs. 256), so the concatenated vector is dominated by the search dimensions. In the Euclidean space where RQ-KMeans operates, the variance contributed by the search dimensions is approximately 75% of the total, making the quantization centroids primarily capture search-relevant structure at the expense of recommendation-relevant structure.

This is a concrete failure mode of post-hoc fusion: even though both embedding sources are present in the combined vector, the structural properties of that vector (dimensionality ratios, variance distribution) cause one signal to dominate the quantization outcome.

FusedSVD IDs

Strategy. This approach addresses the dimensionality imbalance of Fusedconcat by first equalizing the embedding dimensionalities:

Apply $\ell_2$ -normalization to both $\mathbf{v}_{\text{search}}$ and $\mathbf{v}_{\text{rec}}$ .
Reduce the higher-dimensional search embedding from 768 to 256 dimensions using truncated SVD (Singular Value Decomposition), keeping the top 256 singular vectors. This projects $\mathbf{v}_{\text{search}} \in \mathbb{R}^{768}$ down to $\mathbf{v}_{\text{search}}^{\text{reduced}} \in \mathbb{R}^{256}$ .
Element-wise sum the equal-dimensional embeddings:

$\mathbf{v}_{\text{svd}} = \mathbf{v}_{\text{search}}^{\text{reduced}} + \mathbf{v}_{\text{rec}}$

where both vectors are now in $\mathbb{R}^{256}$ . This combined vector is quantized with RQ-KMeans to produce a shared Semantic ID.

Design rationale. The SVD reduction ensures both embedding spaces have equal influence on the final vector by giving them equal dimensionality. Summation (rather than concatenation) produces a compact 256-dimensional vector that encodes information from both sources in a shared subspace. The intuition is that the SVD retains the principal directions of variation in the search embedding space — the dimensions that capture the most information about search relevance — while discarding less informative directions, making room for the recommendation signal to contribute equally.

What the results show (Section 5). FusedSVD achieves search R@30 of 0.033 and recommendation R@30 of 0.038. Compared to Fusedconcat, recommendation performance improves substantially (from 0.018 to 0.038), but search performance drops (from 0.048 to 0.033). The equalization works — it shifts the balance toward recommendation — but the overall effectiveness on both tasks is below what is achieved by the Multi-task approach. This suggests that dimensionality equalization is necessary but not sufficient for good joint performance. Truncated SVD discards information from the search embedding (the lower 512 singular vectors), and summation assumes the two embedding spaces can be meaningfully combined in a shared 256-dimensional subspace — an assumption that may not hold if the subspaces encoding search relevance and collaborative filtering are largely orthogonal.

Another solution the paper mentions (Section 5, footnote 4) but does not explore is training embeddings with a Matryoshka objective (Kusupati et al., 2022), where the model is trained to produce good representations at multiple dimensionalities (e.g., the first 256 dimensions alone are optimized to be useful). This would naturally produce search and recommendation embeddings at equal dimensionality without post-hoc SVD.

Multi-task IDs

Strategy. This is the approach the paper advocates as the best trade-off. Instead of training separate embedding models and then fusing their outputs, train a single bi-encoder on both supervision signals simultaneously, as described in the "Multi-task bi-encoder embedding" subsection above. The output $\mathbf{v}_{\text{mt}} \in \mathbb{R}^{768}$ is directly quantized using RQ-KMeans (two codebooks of size 256) to produce a shared Semantic ID $\langle c_1, c_2 \rangle$ used for both tasks.

What distinguishes Multi-task from Fused approaches. The critical difference is when the combination happens. In Fusedconcat and FusedSVD, two embedding models are trained independently on separate objectives, and their outputs are combined after training. The embedding models never see each other's gradients; they optimize for their respective tasks in isolation. In Multi-task, a single model is trained with gradients from both objectives flowing through shared parameters. This forces the model to learn representations that must satisfy both loss terms simultaneously during training, creating a jointly optimized embedding space rather than a post-hoc composite of independent spaces.

Why joint training outperforms post-hoc fusion. The paper's results (Table 2) show Multi-task achieves search R@30 of 0.046 and recommendation R@30 of 0.049 — the best balanced performance among all methods, and competitive with or exceeding all other cross-task approaches on both tasks. The authors do not provide a mechanistic explanation for why joint training is better, but the empirical pattern suggests several possible factors:

Shared representations learn to encode information relevant to both tasks, not just one. During training, the encoder must represent items such that both query–item and item–item similarities are captured. This encourages the model to discover latent dimensions that correlate with both signals — for instance, a dimension capturing "broad audience appeal" might help predict both search relevance (popular items match more diverse queries) and recommendation co-occurrence (popular items appear in many histories).
Gradient interference may act as regularization. When the two objectives conflict for a specific item (a niche film with very specific search queries but also a dedicated audience that watches similar films), the opposing gradients prevent overfitting to either signal alone, producing a more robust representation that generalizes better to the joint evaluation setting.
No post-hoc dimensionality reduction or balancing. Unlike Fusedconcat (dimensionality imbalance) and FusedSVD (information loss from SVD projection and the assumption of additive subspaces), Multi-task avoids these post-hoc artifacts by letting the training process itself determine how to allocate the embedding dimensions to the two tasks.

Token budget comparison. Multi-task uses the same number of tokens as the single-task baselines (512), half of Separate (1024), and fewer than Prefix-share (768). This makes it the most parameter-efficient approach among cross-task methods.

What happens to popularity bias (Table 2). The Multi-task embeddings produce recommendation results that vary by popularity: Head R@30 of 0.135 and Torso R@30 of 0.024. The Head performance is strong (second only to the recommendation-tuned baseline at 0.170), while Torso performance is lower than the search-tuned baseline (0.070). This suggests the Multi-task encoder learns to rely on collaborative filtering patterns (which are strong for popular items with rich interaction histories) for Head items and on content signals (which are weaker but available for all items) for Torso items. For less popular items, collaborative filtering is sparse, and the content-grounding from the search loss provides a fallback — but a less effective one than the pure recommendation-tuned embedding, which can still capture collaborative patterns even for mid-popularity items through matrix factorization.

Quantization (ID Tokenization)

The quantization step maps continuous embedding vectors to discrete token sequences. All embedding construction strategies (except Prefix-share, which requires its own auto-encoder quantizer) use the same quantization method: Residual Quantization with K-Means (RQ-KMeans).

What RQ-KMeans does. Given an embedding $\mathbf{v} \in \mathbb{R}^d$ and $K$ codebooks each of size $V$ :

Codebook 1. Find the centroid $\mathbf{c}^{(1)}_j$ in the first codebook (containing $V$ centroids in $\mathbb{R}^d$ ) that is closest to $\mathbf{v}$ in Euclidean distance. Record the index $c_1 = j$ . Compute the residual: $\mathbf{r}_1 = \mathbf{v} - \mathbf{c}^{(1)}_{c_1}$ .
Codebook 2. Find the centroid in the second codebook that is closest to the residual $\mathbf{r}_1$ . Record $c_2$ and compute $\mathbf{r}_2 = \mathbf{r}_1 - \mathbf{c}^{(2)}_{c_2}$ .
Continue for all $K$ codebooks.

The final Semantic ID is $\langle c_1, c_2, \ldots, c_K \rangle$ , and the reconstruction is $\hat{\mathbf{v}} = \sum_{k=1}^K \mathbf{c}^{(k)}_{c_k}$ .

Why residual quantization. Ordinary K-Means with a single codebook of size $V^K$ (to produce $K$ -token sequences) would require storing $V^K$ centroids — with $V=256$ and $K=2$ , that is 65,536 centroids, which is memory-intensive and computationally expensive to search. Residual quantization decomposes this into $K$ codebooks of size $V$ each, requiring only $K \cdot V = 512$ centroids total, while still producing $V^K = 65{,}536$ possible combinations. Each codebook captures successively finer residual structure: the first codebook captures the coarse structure of the embedding space (the 256 most important "regions"), and the second codebook captures the residual variation within each region.

Implementation. The paper uses FAISS's residual quantizer implementation (Douze et al., 2024). The codebooks are learned by running K-Means on the embedding vectors of all 62,138 movies in the dataset. After training, each item's embedding is quantized by greedily finding the nearest centroid at each step (no beam search over codebook combinations — the standard RQ-KMeans greedy assignment).

Tokenization method ablation (Table 3). To verify that the findings are not artifacts of the specific quantization algorithm, the paper ablates four tokenization methods, keeping the embedding space fixed to the Multi-task embeddings:

RQ-KMeans: The primary method described above. Achieves search R@30 of 0.046 and recommendation R@30 of 0.049.
Dictionary encoding (MiniBatchDictionaryLearning): A sparse coding approach from scikit-learn (Pedregosa et al., 2011) that learns a dictionary of basis vectors and represents each embedding as a sparse combination of atoms. The atom indices with the highest coefficients are used as tokens. Achieves search R@30 of 0.019 and recommendation R@30 of 0.029.
ResidualLFQ (Lookup-Free Quantization): A variant of residual quantization that uses a fixed quantization grid (no learned centroids), simplifying the codebook to a structured lattice. Achieves search R@30 of 0.018 and recommendation R@30 of 0.023.
RQ-VAE (Residual Quantized Variational Autoencoder): Trains an encoder-decoder architecture where the bottleneck is a residual quantizer, with a commitment loss that encourages the encoder to produce representations close to codebook centroids. This is the method used by Prefix-share (and is the standard approach in much prior work on Semantic IDs, e.g., Rajput et al., 2023). Achieves search R@30 of 0.002 and recommendation R@30 of 0.024.

Why RQ-KMeans outperforms learned auto-encoder approaches. The paper notes that Hong et al. (2025) "also found RQ-VAE unstable and opted for hierarchical k-means." The authors do not provide a detailed analysis of why, but the likely reasons include:

Training instability in RQ-VAE. The commitment loss and the reconstruction loss compete — the encoder must produce representations that are simultaneously good for reconstruction and close to codebook centroids. Balancing these losses (especially with multiple quantization layers) is notoriously tricky, and the codebook collapse problem (where most items map to a small subset of centroids, wasting the remaining codebook capacity) is well-documented.
Greedy K-Means assignment is simpler. RQ-KMeans does not require training an encoder-decoder; it directly clusters the fixed embedding vectors. There is no distribution shift between training and inference, no commitment loss tuning, and no risk of codebook collapse. The trade-off is that the embeddings are not optimized for quantizability (unlike in RQ-VAE, where the encoder learns to produce "quantization-friendly" representations), but for the Multi-task embeddings — which are already well-structured from the contrastive training — the greedy K-Means assignment appears to be sufficient.

Implications for the main findings. Because RQ-KMeans consistently outperforms alternatives, the paper's conclusions about Multi-task embeddings are robust to the choice of tokenizer. The Multi-task embedding advantage is not an artifact of a particular quantization method working better with joint embeddings — it holds (and is in fact amplified) under the best-performing tokenizer.

Joint Generative Model Training and Inference

The final component is the generative model that maps search and recommendation prompts to Semantic ID token sequences. This model is held constant across all embedding construction strategies to isolate the effect of the Semantic ID design.

Model architecture. The paper uses google/flan-t5-base (Raffel et al., 2020), an encoder-decoder transformer with approximately 250 million parameters, pre-trained on a mixture of supervised and unsupervised text-to-text tasks (the "Flan" instruction-tuning procedure). The choice of Flan-T5-base, rather than a larger model, reflects the paper's focus on the embedding and ID construction pipeline rather than on scaling the generative model. The base size is sufficient to demonstrate the relative differences between ID construction strategies while keeping training tractable.

Vocabulary expansion. The generative model's vocabulary is expanded to include the new Semantic ID tokens — the codebook-specific tokens that represent centroid indices. For the standard configuration (two codebooks of size 256), this adds 512 tokens. The embeddings for these new tokens are randomly initialized and learned during the 3-epoch joint training. This is a standard approach in generative retrieval: the model learns to map between natural language prompts and item ID tokens through the shared transformer parameters.

Training data and prompts. The model is trained jointly on both search and recommendation data, interleaved in batches. The prompts are formatted as task-specific text prefixes:

Search: "Search: {query}" — where {query} is one of the 10 training queries per item, and the target is the Semantic ID of the relevant item.
Recommendation: "Recommend: {user interaction history}" — where {user interaction history} is a sequence of item Semantic IDs that the user has previously interacted with, and the target is the Semantic ID of the next item. The paper does not specify the exact format of the interaction history (e.g., how many previous items are included, whether they are delimited by special tokens), but the standard approach in generative sequential recommendation (e.g., Petrov and Macdonald, 2023; Qu et al., 2024) is to represent the history as a space-separated or comma-separated sequence of previously interacted item IDs, ordered chronologically.

Training hyperparameters (Section 4, "Generative model"). The Flan-T5-base model is trained for 3 epochs with learning rate 0.002, batch size 128, using the AdamW optimizer with weight decay 0.01. These are relatively standard fine-tuning hyperparameters for T5 models. The 3-epoch duration suggests each item is seen a limited number of times during training, which is typical for generative retrieval where the model must memorize item-to-ID mappings in addition to learning retrieval patterns.

Training objective. The model is trained with standard autoregressive cross-entropy loss: for each target Semantic ID sequence $y = \langle y_1, y_2, \ldots, y_K \rangle$ :

$\mathcal{L}_{\text{gen}} = -\sum_{t=1}^K \log P(y_t \mid y_{<t}, \text{prompt})$

where $P(y_t \mid y_{<t}, \text{prompt})$ is the model's predicted probability for the correct token at position $t$ given the prompt and all previous target tokens. This is the same loss used for standard language modeling and sequence-to-sequence tasks.

Inference: Diversified beam search. At inference time, the model receives a search query or recommendation prompt and must generate a set of candidate item Semantic IDs. To increase the number of distinct items retrieved (since beam search tends to produce similar sequences that may decode to the same or similar items), the paper uses diversified beam search (Vijayakumar et al., 2016):

Beam size 60: The decoder maintains 60 candidate partial sequences at each decoding step.
Diversity penalty 0.25: A penalty term is subtracted from the score of a candidate if it shares tokens with previously decoded sequences in the same group. This encourages the beam to explore diverse regions of the output space, producing distinct Semantic IDs that map to different items rather than converging on the most likely ID and its near neighbors.
30 groups: The beam is partitioned into 30 groups, and diversity penalties are applied within each group. This controls the granularity of the diversity encouragement.

The result is up to 30 distinct item IDs ranked by the beam search scores (after diversity ranking). The paper evaluates Recall@30 — the fraction of test instances where the correct item appears among the top-30 retrieved items. This metric directly measures whether the generative model can surface the relevant item within a retrieval budget of 30 candidates.

Why diversified beam search is necessary for evaluation. Standard beam search with a large beam width tends to produce many near-duplicate sequences that decode to the same Semantic ID (since small variations in token probabilities can lead to the same codebook indices). Recall@30 would be artificially low if the 30 decoded sequences correspond to only 5–10 distinct items. The diversity penalty and group structure ensure that the model's output truly covers up to 30 different items, making the Recall@30 metric meaningful.

Generalization to unseen items. A practical advantage of Semantic IDs noted in the paper (Section 1) is cold-start capability. Because Semantic IDs are constructed from an item's embedding, a new item not seen during generative model training can still be represented and retrieved — as long as its embedding falls into the same codebook regions as training items (so its Semantic ID is composed of known tokens) and the generative model has learned to associate those token combinations with relevant prompts. The paper does not explicitly evaluate cold-start performance, but the embedding-based construction (rather than memorizing fixed ID assignments) is what enables this property.

Summary of Design Choices and Their Justifications

Multi-task bi-encoder over separate task-specific encoders: Joint training allows gradients from both objectives to shape a shared representation space, avoiding the information loss and dimensionality balancing problems of post-hoc fusion (Fusedconcat, FusedSVD) and the cross-task isolation of token separation (Separate).
Contrastive loss with in-batch negatives over explicit negative sampling: The MultipleNegativesRankingLoss (for search) and item–item pair contrastive loss (for recommendation) are computationally efficient — they use all other items in the batch as negatives without needing a separate negative sampling step. This scales well with batch size (512) and simplifies the training pipeline.
ENMF for collaborative filtering embeddings over other CF models: ENMF is chosen because it is computationally efficient (no negative sampling required, unlike BPR-based matrix factorization) and produces item embeddings of moderate dimensionality (256), making it integrated with the bi-encoder embeddings for the fusion experiments.
RQ-KMeans over RQ-VAE for quantization: RQ-KMeans is simpler (greedy K-Means assignment), more stable (no encoder-decoder training, no commitment loss), and empirically outperforms RQ-VAE and other learned quantization approaches (Table 3). The ablation validates that this choice does not bias the comparison between embedding strategies.
Two codebooks of size 256 over larger codebooks: This balances token budget (512 new tokens), representation capacity (65,536 possible Semantic IDs, sufficient for 62,138 items with room for generalization), and training efficiency (smaller vocabulary means fewer parameters and faster training). Larger codebooks would increase capacity but also increase model size and training time.
Diversified beam search (beam 60, 30 groups) over standard beam search or sampling: Standard beam search with large beam width produces many near-duplicate item IDs, reducing effective recall. Diversified beam search ensures the 30 output sequences correspond to distinct items, making Recall@30 a meaningful metric. The configuration (beam 60, 30 groups) is chosen to produce approximately 30 distinct items while maintaining sufficient exploration (beam 2× the number of groups).
Flan-T5-base over larger generative models: The paper's focus is on comparing Semantic ID construction strategies, not on pushing absolute performance. Flan-T5-base (250M parameters) is large enough to demonstrate meaningful performance differences between strategies while being small enough for rapid experimentation (3 epochs on MovieLens25M). The relative ranking of strategies is expected to generalize to larger models, though the absolute performance would likely improve.
Synthetic queries for search data over real search logs: MovieLens25M lacks real search queries. Generating 10 training and 10 test queries per movie with Gemini-2.0-flash creates a controlled search dataset where every item has equal query coverage (no popularity bias in search). This is acknowledged as a limitation — real search distributions likely correlate with recommendation popularity, which would make the Multi-task approach even more effective — but it ensures a fair comparison where test set difficulty is uniform across items.

4. Key Insights and Innovations

Innovation 1: Joint Embedding Training as a Pareto-Dominant Strategy for Multi-Task Item Representation

The paper's most significant conceptual contribution is the demonstration that when and how task objectives are combined during embedding learning fundamentally determines whether a unified Semantic ID space can serve multiple retrieval tasks. This is not an obvious finding — the natural engineering intuition, reflected in prior work and in the paper's own Separate and Fused baselines, is that task specialization should be preserved somehow: either by giving each task its own representation space (Separate, Prefix-share) or by carefully combining independently optimized representations after the fact (Fusedconcat, FusedSVD). The paper shows that both intuitions are wrong for this setting, but for different and instructive reasons.

What the field assumed before this work. The dominant assumption in generative retrieval — visible in RIPOR (Zeng et al., 2024), TokenRec (Qu et al., 2024), TIGER (Rajput et al., 2023), and DSI (Tay et al., 2022) — was that Semantic IDs should be constructed from embeddings optimized for the target task. This made sense for single-task systems: if you're building a search model, use search-tuned embeddings; if you're building a recommender, use collaborative-filtering embeddings. When the field began exploring joint search-and-recommendation models (Penha et al., 2024; Shi et al., 2025), the implicit assumption carried over: you need task-specific information encoded somewhere in the ID scheme. The question was how to combine task-specific signals — through separate tokens, shared prefixes, or embedding fusion — not whether task-specific signals were needed.

The concurrent work by Shi et al. (2025), which proposes Prefix-share (shared codebook plus task-specific codebooks), exemplifies this assumption. It accepts that task-specific representation capacity is necessary and tries to find the optimal division between shared and task-specific tokens. The paper's Prefix-share results (search R@30 = 0.007, recommendation R@30 = 0.021) show this architecture underperforms substantially, though the authors attribute this to the quantization method rather than the architectural assumption itself.

What this paper demonstrates instead. The Multi-task approach makes a stronger claim: you don't need task-specific representation capacity at all. A single embedding space, trained with gradients from both tasks flowing through shared parameters, produces Semantic IDs that outperform all methods that preserve task-specific structures. Multi-task achieves search R@30 of 0.046 and recommendation R@30 of 0.049 (Table 2) — simultaneously competitive with the search-tuned baseline on search (0.072 vs. 0.046) and the recommendation-tuned baseline on recommendation (0.062 vs. 0.049), without any task-specific tokens or post-hoc fusion. The 0.049 recommendation figure is particularly striking because it nearly matches the recommendation-tuned baseline (0.062) despite the bi-encoder having no collaborative filtering architecture — it sees only item metadata as input and learns collaborative patterns indirectly through the item–item co-occurrence loss.

Why this is a conceptual shift, not just a better method. The paper's finding implies that the representation tension between search and recommendation — which Section 2 establishes as the core problem — is not an irreducible conflict that requires task-specific representations to resolve. Instead, it is a tension that can be resolved during representation learning itself if the embedding model is trained on both objectives. The shared encoder discovers dimensions that serve both masters, and — crucially — this jointly optimized space produces Semantic IDs that work better for both tasks than any approach that trains task-specific encoders and then tries to compose their outputs.

This is a fundamental result about multi-task representation learning in discrete retrieval systems, not an incremental improvement. It suggests that the representation collision identified in the paper is an artifact of training embeddings separately, not an inherent property of the tasks. When the same parameters must satisfy both objectives during training, they discover a compromise that is qualitatively different from — and superior to — any post-hoc combination of independently optimized representations.

The Fused results provide the diagnostic evidence for this claim. Fusedconcat (search 0.048, recommendation 0.018) and FusedSVD (search 0.033, recommendation 0.038) both underperform Multi-task on at least one task, despite having access to the same embedding information (the search-tuned and recommendation-tuned vectors). The difference is not the information content — it's that in Multi-task, the embedding model learned to represent that information jointly from the start, avoiding the dimensionality imbalance, information loss, and representational incompatibility that plague post-hoc fusion.

Limits of the claim. The paper does not establish whether this finding generalizes beyond the specific task pair (search + recommendation), the specific embedding architecture (bi-encoder with contrastive loss), or the specific domain (movies with synthetic queries). It is possible that for more divergent tasks — say, search and dialogue generation — joint embedding training would fail and task-specific representations would be necessary. But within the scope of retrieval tasks that share an item catalog, the finding is a clear challenge to the prevailing assumption that task specialization in the ID space is required.

Innovation 2: The Token Separation Trap — Why Distinct Task Vocabularies Backfire in Joint Models

A second distinctive finding is the failure of token-level task separation as a strategy for handling competing objectives. The Separate approach — give each task its own Semantic ID tokens from its own embedding space — seems like the most natural solution to the representation collision problem. If search and recommendation need different item similarities, let them have different item representations. The generative model can learn to route between them based on the task prompt.

The paper shows this fails (search R@30 = 0.028, recommendation R@30 = 0.032), and the failure is instructive because it reveals something non-obvious about how generative models learn from shared vocabularies.

What the field assumed. The intuition behind token separation draws on a broader pattern in multi-task NLP: giving tasks their own output heads or vocabulary segments prevents interference and allows specialization. In classification, multi-task models often use task-specific output layers. In text generation, task-specific control tokens are standard. Separate applies this logic to Semantic IDs: search tokens for search, recommendation tokens for recommendation, no interference.

Concurrent work (Shi et al., 2025) with Prefix-share refines this intuition — share some tokens, keep others task-specific — but preserves the assumption that some degree of token-level separation is beneficial.

What the data shows. Separate performs substantially worse than all embedding-combined approaches (Fusedconcat, FusedSVD, Multi-task) on both tasks, and worse than the single-task baselines. This means that having two different ways to represent the same item actively harms the generative model's ability to learn either task well.

The missing mechanism: cross-task regularization through shared tokens. The paper's explanation — drawing on Penha et al. (2024) — is that "the knowledge learned from one task cannot be used for the task-specific tokens of the other task, negating the regularization effect in item representations." This requires unpacking.

When the generative model is trained jointly on search and recommendation, its encoder processes both types of prompts and builds internal representations of items. If search and recommendation share the same Semantic ID tokens for an item, the decoder's token embeddings learn to represent properties of that item that are useful for both tasks — because the same embedding must help predict the item for both search queries and recommendation contexts. This creates a cross-task pressure on the token embeddings: they must encode item properties that generalize across tasks.

When search and recommendation use separate token vocabularies, this pressure disappears. The search tokens only need to help predict items for search queries; the recommendation tokens only need to help predict items for recommendation contexts. The decoder can learn completely different representations for the two token sets, and there is no gradient signal pushing them to share structure. The knowledge that Item A is popular, frequently searched, and often co-consumed with Item B — knowledge that the encoder might build up from seeing both types of prompts — has no pathway to influence both output distributions because the output embeddings are disjoint.

This is not just a parameter inefficiency (doubling the token budget). It is a learning dynamics failure: the joint training regime, which should allow each task to benefit from patterns in the other's data, is stripped of its mechanism for doing so. The Separate model is effectively two single-task models sharing a transformer backbone, with minimal cross-task transfer.

Why the finding is non-obvious. The standard concern with shared vocabularies for multi-task models is interference — the same token embedding being pulled in different directions by different tasks. The paper's result inverts this concern: in the joint S&R setting, the absence of shared tokens is the larger problem, because it prevents beneficial regularization. The interference that shared tokens might cause (Item A's search-relevant properties confusing its recommendation-relevant properties) is apparently less damaging than the isolation that separate tokens enforce.

This has implications beyond Semantic IDs. It suggests that for any multi-task generative model where tasks operate on the same set of entities (items, products, documents), sharing the entity representation tokens is not merely a parameter-saving convenience — it may be necessary for the model to learn task-general entity representations that transfer knowledge between tasks. Token separation is an anti-pattern in this regime, not a neutral design choice.

Quantization as a confound. The Prefix-share result (search 0.007, recommendation 0.021) complicates the picture because Prefix-share does share some tokens (the shared codebook) but performs even worse than Separate. The paper attributes this to the quantization method (auto-encoder quantizers underperforming RQ-KMeans), not to the token-sharing logic itself. This means the negative result for token separation is cleanest for the Separate vs. embedding-combined comparison, where the quantization method is held constant (all use RQ-KMeans). The Separate failure is attributable to the token separation strategy itself, not to any confounded quantization choice.

Innovation 3: Dimensionality Imbalance as a Hidden Failure Mode in Post-Hoc Embedding Fusion

The paper identifies and diagnoses a specific, non-obvious failure mode in post-hoc embedding fusion: dimensionality imbalance causes one embedding space to dominate the quantization outcome, even when both are $\ell_2$ -normalized and concatenated.

What the data shows. Fusedconcat — which concatenates the 768-dimensional $\mathbf{v}_{\text{search}}$ and 256-dimensional $\mathbf{v}_{\text{rec}}$ into a 1024-dimensional vector before quantization — produces results heavily skewed toward search (search R@30 = 0.048, recommendation R@30 = 0.018). Recommendation performance is barely above the search-only baseline (0.026) and far below the true recommendation capability (0.062). FusedSVD — which equalizes dimensionalities to 256 via truncated SVD on the search embedding before summation — shifts the balance back toward recommendation (search 0.033, recommendation 0.038), but search performance drops.

Why this is a diagnostic contribution, not just a hyperparameter tuning result. The dimensionality imbalance problem is not obvious a priori. A natural assumption when concatenating normalized embeddings is that both contribute equally to the resulting distance metric — after $\ell_2$ -normalization, each vector has unit norm, so the concatenated vector has norm $\sqrt{2}$ regardless of the individual dimensionalities. However, the Euclidean distance used by RQ-KMeans depends on the distribution of variance across dimensions, not just the overall norm. The search embedding's 768 dimensions collectively contribute roughly 75% of the total variance in the concatenated space (768/(768+256) = 0.75), meaning the quantization centroids are primarily determined by search-relevant structure. The recommendation signal is present in the concatenated vector — it has not been thrown away — but it is swamped by the search signal in the distance computations that drive codebook assignment.

FusedSVD's partial recovery of recommendation performance — and its corresponding loss of search performance — confirms this diagnosis. When the search embedding is projected down to 256 dimensions, it loses information (the lower 512 singular vectors are discarded), but its influence is now balanced with the recommendation embedding. The trade-off is explicit: equal dimensionality enables both signals to influence the quantization, but at the cost of compressing the richer search representation.

Connection to broader ML practice. Dimensionality imbalance in concatenated feature vectors is a common pitfall in multi-modal representation learning. When features from different sources have different native dimensionalities, simple concatenation implicitly weights the higher-dimensional source more heavily in any distance-based downstream operation (clustering, nearest-neighbor search, quantization). The standard remedies — dimensionality reduction (PCA, SVD), learned projections, or weighted concatenation — all involve discarding or downweighting information from the higher-dimensional source. The paper's finding that none of these post-hoc remedies matches joint training from scratch suggests that the problem is not merely dimensional but representational: the embedding subspaces optimized independently for different tasks do not compose well in a shared vector space, regardless of how their dimensionalities are balanced.

This insight has implications for any system that fuses independently trained embeddings for downstream discrete tokenization — which is an increasingly common pattern as practitioners combine pre-trained text embeddings, collaborative filtering embeddings, and other modality-specific representations into unified item tokens for generative models. The paper provides evidence that training a single model on all objectives jointly avoids both the dimensionality imbalance problem and the deeper representational incompatibility problem that dimensionality equalization alone cannot fix.

Innovation 4: The Quantization Simplicity Principle — Why Non-Learned Tokenization Outperforms Learned Alternatives

The paper's tokenization ablation (Table 3) — while presented as a brief ablation rather than a central contribution — contains a finding with implications for the broader generative retrieval literature: simple, non-learned residual quantization (RQ-KMeans) substantially outperforms more sophisticated learned quantization methods (RQ-VAE, ResidualLFQ) for Semantic ID construction.

The data. On Multi-task embeddings, RQ-KMeans achieves search R@30 = 0.046 and recommendation R@30 = 0.049. RQ-VAE — the method used by TIGER (Rajput et al., 2023) and many subsequent works — achieves search R@30 = 0.002 and recommendation R@30 = 0.024. The drop is catastrophic for search (a 23× reduction) and substantial for recommendation (a 2× reduction). ResidualLFQ and dictionary encoding fall in between but still substantially underperform RQ-KMeans.

Why this challenges prevailing assumptions. The generative retrieval literature has largely adopted VAE-based quantization as the default. The intuition is compelling: an auto-encoder can learn to produce embeddings that are inherently quantization-friendly, reducing the reconstruction error compared to clustering fixed embeddings. The encoder is trained jointly with the quantizer, so the representations are optimized for the discrete bottleneck.

The paper's results suggest this intuition breaks down in practice, at least at the scale studied. The authors do not provide a detailed diagnosis — they note that Hong et al. (2025) "also found RQ-VAE unstable and opted for hierarchical k-means" — but the likely factors (codebook collapse, commitment loss tuning difficulty, encoder-quantizer distribution shift) are well-known challenges in the VQ-VAE literature more broadly.

The principle it suggests. The paper does not explicitly frame this as a principle, but the result points to one: when the input embeddings are already well-structured by a contrastive or supervised objective, learned quantization adds complexity without adding value. The Multi-task bi-encoder — trained with two contrastive losses — produces embeddings where semantically similar items are already close in Euclidean space. Clustering these embeddings with simple K-Means works well because the structure that the quantizer needs to capture is already present in the embedding geometry. Adding an encoder-decoder wrapper that learns to "improve" the embeddings for quantization can actually degrade them — either by collapsing the codebook (so most items map to a few centroids, losing discriminability) or by distorting the embedding space to be more quantization-friendly at the expense of task relevance.

This is a simplicity principle for the generative retrieval pipeline: invest complexity in the embedding model (where task-specific objectives can shape the representation) and keep the quantization step as simple as possible (greedy K-Means assignment on frozen embeddings). The paper's results suggest that efforts to improve Semantic ID quality through more sophisticated quantization — a direction pursued in multiple prior works — may be misplaced relative to the simpler alternative of improving the embedding model that feeds the quantizer.

Scope and caveats. The paper tests this on one dataset (MovieLens25M) with one generative model (Flan-T5-base) and one family of embedding architectures (Sentence Transformers). It is possible that at larger scales (millions of items, larger codebooks, larger generative models) the relative performance of learned quantization improves. But as a finding that shifts the burden of proof — learned quantization must now demonstrate it beats simple RQ-KMeans, rather than RQ-KMeans being treated as a weak baseline — it is a useful corrective to current practice.

5. Experimental Analysis

Evaluation Methodology

Dataset. The experiments use MovieLens25M (Harper and Konstan, 2015), containing 62,138 movies and 1.24 million user–item interactions. The data is split chronologically, with the last interaction per user held out for testing. Since MovieLens lacks real search logs, the authors generate 20 natural-language queries per movie using Gemini-2.0-flash, with 10 assigned to training and 10 to testing. The generation prompt instructs the model to produce diverse, realistic queries covering different aspects of each movie without including the movie's title. The prompt also asks for a second set of 10 paraphrased queries, for a total of 20 per item. This yields a search dataset where every item has exactly the same number of queries — a uniform distribution with no popularity bias, which the authors explicitly note diverges from real-world search behavior and may underestimate the benefits of joint embedding training.
Base model(s). The generative model is google/flan-t5-base (Raffel et al., 2020), an encoder-decoder transformer with approximately 250 million parameters, pre-trained on instruction-tuned text-to-text tasks. The embedding models vary by experimental condition: the content-based baseline uses all-mpnet-base-v2 from Sentence Transformers (Reimers and Gurevych, 2019), a 768-dimensional bi-encoder pre-trained on general semantic similarity tasks. Task-specific fine-tuning starts from this same checkpoint for search-tuned and multi-task variants. The recommendation-tuned model is Efficient Neural Matrix Factorization (ENMF; Chen et al., 2020), implemented via RecBole (Zhao et al., 2022), producing 256-dimensional item embeddings. The choice of Flan-T5-base over larger models is deliberate: it is large enough to demonstrate relative differences between ID construction strategies while being small enough for rapid experimentation (3 epochs). The paper's focus is on comparing Semantic ID strategies, not on pushing absolute retrieval performance.
Metrics. The primary metric is Recall@30 (R@30) — the fraction of test instances where the correct item appears among the top 30 retrieved Semantic IDs. During inference, the generative model is provided with a search query or a recommendation prompt and generates candidate item IDs via diversified beam search (beam size 60, diversity penalty 0.25, 30 groups). Recall@30 is computed by checking whether the ground-truth item's Semantic ID (for recommendation) or the item associated with the ground-truth query (for search) is in the top-30 decoded candidates. Every model is run five times with different random seeds, and the mean recall across runs is reported. Statistical significance is assessed using paired Student's t-tests with a 95% confidence interval and Bonferroni correction, indicated by superscripts in Table 1 and Table 2.
Baselines. The paper compares against two task-specific baselines and four cross-task methods (in addition to the Multi-task approach it advocates). The task-specific baselines are: (1) Search-based Semantic IDs — embeddings from a bi-encoder fine-tuned only on search data (query–item pairs), following the approach of RIPOR (Zeng et al., 2024), quantized with RQ-KMeans into two codebooks of size 256; (2) Recommendation-based Semantic IDs — embeddings from an ENMF model trained on user–item interactions, following TokenRec (Qu et al., 2024), quantized identically. These baselines establish single-task upper bounds and demonstrate the cross-task degradation when a task-specialized Semantic ID is used for both tasks. The cross-task baselines include: (3) Separate — task-specific Semantic IDs from both sources, concatenated with task tags (adding 1,024 tokens to the generative model's vocabulary); (4) Prefix-share — three-codebook auto-encoder quantization adapted from Shi et al. (2025), with one shared codebook (256 tokens) and two task-specific codebooks (256 each, totaling 768 new tokens); (5) Fusedconcat — ℓ₂-normalized concatenation of search and recommendation embeddings into a 1,024-dimensional vector, then RQ-KMeans quantized; (6) FusedSVD — dimensionality-equalized summation, where truncated SVD reduces the 768-dimensional search embedding to 256 dimensions before element-wise summation with the 256-dimensional recommendation embedding, then RQ-KMeans quantized. An implicit (7) Content-based baseline (Table 1, row 1) uses the pre-trained all-mpnet-base-v2 embeddings without any task-specific fine-tuning, representing the DSI (Tay et al., 2022) / TIGER (Rajput et al., 2023) approach.
Generation budget / compute accounting. The paper measures computational cost implicitly through the token budget — the number of new tokens added to the generative model's vocabulary for Semantic IDs. The standard configuration uses two codebooks of size 256, adding 512 tokens. Separate adds 1,024 tokens (doubling the budget), and Prefix-share adds 768 tokens. The paper treats token budget as a practical constraint because larger vocabularies increase model parameter count and training time. All other aspects of the pipeline — embedding model training (5 epochs for bi-encoders, 30 epochs for ENMF), generative model training (3 epochs, batch size 128, learning rate 0.002), and inference (diversified beam search with beam 60) — are held constant across conditions. No explicit FLOP or wall-clock time accounting is provided. Importantly, the cost of generating the synthetic queries with Gemini-2.0-flash is not included in any budget calculation.
Cross-validation / statistical protocol. The paper does not use cross-validation for hyperparameter selection or strategy optimization. Instead, all models are evaluated on the fixed chronological test split (last interaction per user). Statistical significance between methods is assessed using paired t-tests across five random seeds with Bonferroni correction (explicitly indicated by superscripts in Table 1, where search-based IDs are marked as significantly better than both content-based and recommendation-based IDs for search, and recommendation-based IDs are marked as significantly better than the others for recommendation). The paper does not report confidence intervals or standard deviations in the main results Table 2, which makes it difficult to assess whether the differences between the top cross-task methods (e.g., Multi-task vs. FusedSVD) are statistically significant. Standard deviations are reported only for the motivation experiments in Table 1.

Main Quantitative Results

Task-Specific Semantic IDs Quantify the Search–Recommendation Trade-off

Table 1 establishes the fundamental tension that motivates the paper. The three rows present the performance of Semantic IDs constructed from increasingly task-specialized embeddings:

Content-based (pre-trained all-mpnet-base-v2, no fine-tuning): Search R@30 = 0.013 (±0.009), Recommendation R@30 = 0.023 (±0.017). This is the lower bound — content similarity alone captures neither search relevance nor collaborative filtering patterns effectively.
Search-based (bi-encoder fine-tuned on query–item pairs): Search R@30 = 0.072 (±0.028), Recommendation R@30 = 0.026 (±0.017). Search performance improves by approximately 5.5× over content-based, but recommendation performance barely moves (+0.003).
Recommendation-based (ENMF collaborative filtering): Search R@30 = 0.004 (±0.001), Recommendation R@30 = 0.062 (±0.015). Recommendation performance improves by approximately 2.7× over content-based, but search performance collapses to near zero.

The pattern is stark and symmetric: optimizing embeddings for one task sacrifices the other. Search-tuned IDs reduce recommendation effectiveness by 58% compared to recommendation-tuned IDs (0.026 vs. 0.062). Recommendation-tuned IDs reduce search effectiveness by 94% compared to search-tuned IDs (0.004 vs. 0.072). The superscripts in Table 1 confirm statistical significance: for search, search-based is significantly better than both content-based (superscript 1) and recommendation-based (superscript 3); for recommendation, recommendation-based is significantly better than both content-based and search-based (superscript 1). The paper contextualizes this finding by noting that "concurrent work reports the same tension" (citing Shi et al., 2025), establishing this as a reproducible phenomenon rather than an artifact of the specific setup.

Cross-Task Semantic ID Construction: Joint Embedding Training Outperforms Post-Hoc Fusion and Token Separation

Table 2 presents the central results of the paper: the performance of five cross-task Semantic ID construction strategies against the two task-specific baselines. The table reports overall R@30 for both tasks, plus a popularity-stratified breakdown for recommendation (Head = top 1% most popular items by training interactions, Torso = remaining 99%).

Task-specific baselines serve as reference points. Search-based achieves Search R@30 = 0.072, Recommendation R@30 = 0.026 (Head: 0.090, Torso: 0.070). Recommendation-based achieves Search R@30 = 0.004, Recommendation R@30 = 0.062 (Head: 0.170, Torso: 0.035). These set the upper bounds for each task and define the Pareto frontier visualized in Figure 1.

Token separation strategies fail. Separate — which allocates distinct Semantic ID vocabularies for search and recommendation by prepending task tags to the respective single-task IDs — achieves Search R@30 = 0.028 and Recommendation R@30 = 0.032 (Head: 0.120, Torso: 0.051). Search performance is 61% below the search-based baseline (0.028 vs. 0.072); recommendation performance is 48% below the recommendation-based baseline (0.032 vs. 0.062). Even compared to the search-based baseline's recommendation performance, Separate only modestly improves recommendation (0.032 vs. 0.026) while severely degrading search. The token budget is doubled to 1,024 new vocabulary entries.

Prefix-share — the three-codebook auto-encoder approach with shared and task-specific tokens — performs worse still: Search R@30 = 0.007, Recommendation R@30 = 0.021 (Head: 0.058, Torso: 0.010). Search is near the recommendation-based baseline level (0.007 vs. 0.004); recommendation is below the search-based baseline (0.021 vs. 0.026). The token budget is 768. The paper attributes this underperformance to the quantization method rather than the architecture itself, as discussed in the tokenization ablation below.

Embedding-level fusion strategies partially resolve the tension. Fusedconcat (ℓ₂-normalized concatenation of 768-dimensional search and 256-dimensional recommendation embeddings into 1,024 dimensions, RQ-KMeans quantized) achieves Search R@30 = 0.048 and Recommendation R@30 = 0.018. This is search-heavy — search performance reaches 67% of the search-based upper bound, but recommendation performance is only 29% of the recommendation-based upper bound and actually lower than the search-based baseline's recommendation figure (0.018 vs. 0.026). The popularity breakdown shows Head R@30 = 0.045 and Torso R@30 = 0.041 — both low, with no Head advantage.

FusedSVD (truncated SVD reduces search embedding to 256 dimensions, then element-wise summation with 256-dimensional recommendation embedding) shifts the balance: Search R@30 = 0.033, Recommendation R@30 = 0.038 (Head: 0.105, Torso: 0.060). Compared to Fusedconcat, recommendation improves by 2.1× (0.018 → 0.038) while search drops by 31% (0.048 → 0.033). Head recommendation (0.105) shows a substantial gain over Torso (0.060), suggesting the collaborative filtering signal from the ENMF embedding is better preserved for popular items.

Multi-task achieves the best balance. The Multi-task bi-encoder — trained jointly on both search (query–item) and recommendation (item–item co-occurrence) contrastive losses, then RQ-KMeans quantized — achieves Search R@30 = 0.046 and Recommendation R@30 = 0.049 (Head: 0.135, Torso: 0.024). This is the best recommendation performance among all cross-task methods and the second-best search performance (after Fusedconcat's 0.048). Crucially, Multi-task is within 11% of the search-based baseline on recommendation (0.049 vs. 0.062 baseline) while maintaining 64% of search-based baseline on search (0.046 vs. 0.072 baseline) — and it does so with the standard 512-token budget, half that of Separate and two-thirds that of Prefix-share.

Popularity-stratified analysis reveals a Head–Torso trade-off. The recommendation performance breakdown in Table 2 reveals that Multi-task's strong overall recommendation figure comes primarily from Head items (R@30 = 0.135, second only to the recommendation-based baseline's 0.170), while Torso items fare worse (R@30 = 0.024, below the search-based baseline's 0.070 and FusedSVD's 0.060). The paper interprets this as Multi-task learning to rely on collaborative filtering signals (strong for popular items with rich interaction histories) for Head items, while falling back on content signals for Torso items — but the content fallback is weaker than the pure collaborative filtering embedding for mid-popularity items. This suggests the Multi-task encoder trades off Torso recommendation quality for balanced search–recommendation performance overall.

The Pareto frontier visualization. Figure 1 plots these results in a 2D space (Search R@30 on the x-axis, Recommendation R@30 on the y-axis). Search-based and Recommendation-based define the extremes of the frontier. Multi-task sits in the upper-right region, dominating both Separate and Prefix-share and being approximately Pareto-optimal (neither Search-based nor Recommendation-based strictly dominates it — Search-based trades lower recommendation for higher search, Recommendation-based trades lower search for higher recommendation). The paper frames Multi-task as the point on the frontier that provides the best trade-off without task-specific specialization.

Robustness across five random seeds. The paper reports means across five seeds. While standard deviations are not provided in Table 2, Table 1's standard deviations give a sense of variability: for the search-based baseline, search R@30 standard deviation is 0.028 around a mean of 0.072, suggesting substantial run-to-run variance that could overlap the differences between top cross-task methods. The significance testing (subscripts in Table 1) confirms only the large gaps — the smaller gaps between Multi-task, Fusedconcat, and FusedSVD in Table 2 may not be statistically distinguishable with the five-seed protocol, though no explicit significance markers are provided for Table 2.

Ablation Studies and Robustness Checks

Tokenization method (Table 3): Ablating the quantization algorithm while keeping the Multi-task embeddings fixed shows that RQ-KMeans is the dominant tokenizer. RQ-KMeans achieves Search R@30 = 0.046, Recommendation R@30 = 0.049. Dictionary encoding (MiniBatchDictionaryLearning from scikit-learn; Pedregosa et al., 2011) achieves Search R@30 = 0.019 and Recommendation R@30 = 0.029 — a 59% drop in search and 41% drop in recommendation. ResidualLFQ (Lookup-Free Quantization with a fixed structured lattice) achieves Search R@30 = 0.018 and Recommendation R@30 = 0.023. RQ-VAE (auto-encoder-based residual quantization with commitment loss, the method used by TIGER and standard in much prior work) achieves Search R@30 = 0.002 and Recommendation R@30 = 0.024 — a 96% drop in search and 51% drop in recommendation compared to RQ-KMeans. The paper notes this finding is consistent with Hong et al. (2025), who "also found RQ-VAE unstable and opted for hierarchical k-means in their experiments." The authors state that similar results were found when performing the same ablation with other embedding approaches, though these data are not shown. This ablation is critical for two reasons: (1) it ensures the main findings are not artifacts of the quantization method — RQ-KMeans is the best tokenizer across the board, so the comparisons between embedding strategies in Table 2 are fair; (2) it explains Prefix-share's underperformance — Prefix-share relies on auto-encoder-based quantization (RQ-VAE or ResidualLFQ), and those methods are inherently weaker than RQ-KMeans in this setting, independent of the prefix-sharing architecture.

Content-based baseline (Table 1, row 1): The content-based row in Table 1 serves as an implicit ablation of task-specific fine-tuning. Removing all fine-tuning (using pre-trained all-mpnet-base-v2 embeddings only) reduces search R@30 from 0.072 to 0.013 and recommendation R@30 from 0.062 to 0.023. This 5–6× degradation confirms that task-specific fine-tuning of embeddings is necessary for competitive retrieval performance — content similarity alone is insufficient. However, it also shows that content-based embeddings are not useless: recommendation R@30 of 0.023 is not zero and is comparable to the search-based baseline's recommendation performance (0.026). This finding is revisited in the discussion of Torso items, where content-based signals provide a fallback when collaborative filtering is sparse.

Embedding fusion method (Fusedconcat vs. FusedSVD, Table 2): The comparison between Fusedconcat and FusedSVD serves as an ablation of dimensionality imbalance handling in post-hoc embedding fusion. Fusedconcat (no dimensionality equalization, 1,024-dimensional vector dominated by 768 search dimensions) is search-heavy (0.048 search, 0.018 recommendation). FusedSVD (SVD reduction to 256 dimensions, element-wise sum) is recommendation-heavy (0.033 search, 0.038 recommendation). This demonstrates that dimensionality imbalance is a real and consequential factor in fusion-based approaches — the higher-dimensional embedding space dominates the quantization outcome. However, even after correction, the fused approaches underperform Multi-task, which avoids the fusion problem entirely by training a single encoder on both objectives.

Popularity stratification (Table 2, Head/Torso columns): The recommendation performance breakdown by popularity reveals how different embedding strategies allocate representational capacity across the item popularity distribution. The recommendation-based baseline is strongest on Head items (0.170) but weak on Torso (0.035) — collaborative filtering excels for popular items with rich interaction data but struggles for less popular items with sparse signals. The search-based baseline shows a smaller Head–Torso gap (0.090 vs. 0.070) — content similarity helps for all items, though absolute performance is low. Multi-task inherits the collaborative filtering's Head strength (0.135, close to the 0.170 baseline) but drops sharply on Torso (0.024, below the search baseline's 0.070 and FusedSVD's 0.060). FusedSVD shows a moderate gap (0.105 Head, 0.060 Torso), suggesting the dimensionality-balanced addition of search and recommendation embeddings provides more robust Torso representations than Multi-task's joint training — a finding that complicates the narrative that Multi-task is uniformly superior and suggests a possible Head–Torso trade-off in joint embedding training.

Negative result: ReST optimization (implied by the paper's framing): The paper does not run an explicit ReST $^{EM}$ or reinforcement learning ablation on the recommendation embeddings, but the Multi-task approach itself represents an implicit negative result for further task-specific optimization — attempting to improve the Multi-task encoder with additional epochs or stronger task-specific losses would risk pushing it toward the extremes seen in the single-task baselines, where better performance on one task comes at the expense of the other. The paper does not explore this directly, leaving open the question of whether Multi-task could be further improved without sacrificing balance.

Critical Assessment

Do the Experiments Support the Claim That Joint Embedding Training (Multi-Task) Provides an "Effective Trade-off" for Joint S&R?

The central claim of the paper — that Multi-task embedding training provides the best balance between search and recommendation performance — is empirically supported by the data in Table 2 and Figure 1. Multi-task achieves Search R@30 = 0.046 and Recommendation R@30 = 0.049. It dominates Separate (0.028/0.032), Prefix-share (0.007/0.021), and Fusedconcat (0.048/0.018) on recommendation while remaining competitive on search. Compared to the Pareto-optimal FusedSVD (0.033/0.038), Multi-task is better on both metrics, though the differences are small in absolute terms (0.013 on search, 0.011 on recommendation).

However, the paper's core framing — that Multi-task provides an "effective trade-off" that challenges "the conventional wisdom that optimal performance requires the construction of a per-task ID" — requires more careful scrutiny.

What "effective trade-off" means in practice. The Multi-task approach leaves substantial performance on the table for both tasks compared to task-specialized IDs. Search R@30 drops from 0.072 (search-based baseline) to 0.046 — a 36% relative reduction. Recommendation R@30 drops from 0.062 (recommendation-based baseline) to 0.049 — a 21% relative reduction. These are non-trivial sacrifices. The claim is not that Multi-task matches task-specialized performance, but that it provides the best single representation for a model that must do both. This claim is supported: among methods that produce one shared Semantic ID per item (Multi-task, Fusedconcat, FusedSVD, Search-based, Recommendation-based), Multi-task is Pareto-optimal, achieving the highest recommendation of any method that also achieves search above 0.040. However, whether this trade-off is "effective" depends on the practitioner's loss function — if search performance is 3× more important than recommendation performance, the search-based baseline (0.072/0.026) may be preferable despite the recommendation gap.

The statistical significance issue. Table 2 does not report standard deviations or significance tests for the cross-task methods. Table 1's standard deviations provide some signal: the search-based baseline has a search standard deviation of 0.028 around a mean of 0.072. If the cross-task methods have similar variance, the differences between Multi-task (0.046), Fusedconcat (0.048), and FusedSVD (0.033) may not be statistically significant. The paper's conclusion that Multi-task "outperforms" these methods could be overstating the case — it may be more accurate to say that Multi-task and Fusedconcat are indistinguishable on search and Multi-task and FusedSVD may overlap on recommendation. The five-seed protocol, while standard, provides limited power for distinguishing small effects.

The missing baseline. A natural question the paper does not address: what happens if you take the best of the two task-specific Semantic IDs on a per-query basis? That is, use the search-based IDs for search queries and the recommendation-based IDs for recommendation prompts. This would require the generative model to route between two different ID spaces based on the task prompt — exactly what Separate attempts but with shared transformer backbone parameters processing both token spaces. The paper shows Separate underperforms, but Separate forces the model to use both ID spaces for both tasks during joint training, which may introduce interference. A cleaner ablation would train the generative model with search-based IDs only for search prompts and recommendation-based IDs only for recommendation prompts (no joint training with the other task's tokens), then evaluate each task independently. This would decouple the generative model's cross-task regularization from the ID construction question and establish whether task-specific IDs + task-specific generative training is actually the upper bound, or whether joint training of the generative model itself (as done here) provides benefits that require shared IDs to realize. The paper does not run this ablation, leaving unclear whether the advantage of shared IDs is due to the ID construction strategy or due to the joint generative model training that the shared IDs enable.

Do the Experiments Support the Claim That Token Separation (Separate) Is Inferior?

The claim that Separate underperforms (Search R@30 = 0.028, Recommendation R@30 = 0.032) is supported by Table 2. However, the paper's explanation — that "the knowledge learned from one task cannot be used for the task-specific tokens of the other task, negating the regularization effect" — is inferred rather than demonstrated. The paper does not provide evidence that cross-task regularization is the mechanism. Alternative explanations include:

Vocabulary size effect. Separate doubles the token budget from 512 to 1,024, which may degrade generative model training at fixed epoch count (3 epochs on MovieLens25M). The model must learn embeddings for more tokens with the same number of gradient steps, potentially leading to undertrained token representations. This confound could be isolated by comparing Separate with increased training epochs against the baseline methods.
Training signal dilution. Each Semantic ID token in Separate is seen only during prompts for its task (search tokens only during search training, recommendation tokens only during recommendation training). The effective training examples per token are halved compared to shared-ID approaches where each token appears in both task contexts. This could explain the performance degradation without invoking cross-task regularization.
The generative model's decoding burden. During diversified beam search inference, the Separate model must search a larger output vocabulary (1,024 additional tokens vs. 512) to find the correct next token. This increases the probability of generating token sequences that produce irrelevant items, reducing recall.

The paper does not isolate these mechanisms, so the claim that token separation prevents beneficial cross-task regularization — while plausible — is an interpretation rather than a demonstrated fact.

Do the Experiments Support the Claim That Dimensionality Imbalance Explains Fusedconcat's Search Bias?

The comparison of Fusedconcat (0.048 search, 0.018 recommendation) and FusedSVD (0.033 search, 0.038 recommendation) provides evidence that dimensionality balancing affects task performance. FusedSVD's SVD projection reduces search embedding dimensionality and improves recommendation performance while degrading search — consistent with the interpretation that the higher-dimensional search embedding dominated Fusedconcat's quantization outcome.

However, the paper does not provide direct evidence for the mechanism. SVD projection does two things simultaneously: it equalizes dimensionality (768 → 256) and it discards information (the lower 512 singular vectors). The drop in search performance could be due to information loss from SVD truncation, not from the removal of dimensionality dominance. An ablation that equalizes dimensionality without information loss — for instance, by padding the recommendation embedding with zeros to 768 dimensions and then concatenating (giving an implicit weighting where recommendation contributes less variance per dimension but equal total dimensions) — would separate these effects. Absent this, the claim that dimensionality specifically causes the imbalance is plausible but not proven.

Furthermore, the paper acknowledges in a footnote that training embeddings with a Matryoshka objective (Kusupati et al., 2022) — where the model produces useful representations at multiple dimensionalities — would be a cleaner solution but does not explore it. This reinforces that the Fused experiments are diagnostic (showing that post-hoc fusion has problems) rather than prescriptive (showing exactly what those problems are and how to fix them).

The Paper's Scope Limitations

Several limitations are intrinsic to the experimental design and constrain the generality of the conclusions:

Single dataset, single domain. All results are on MovieLens25M with synthetic search queries. Movies are a domain where content metadata (title, description, genres, tags) carries substantial information about collaborative similarity (people who watch similar genres tend to watch the same movies). In domains where content signals and collaborative signals are more orthogonal — e.g., e-commerce where product descriptions have limited overlap with purchase co-occurrence patterns, or music where audio features and listening patterns diverge — the Multi-task approach may find it harder to reconcile competing objectives. The paper does not discuss this domain-specificity.
Synthetic queries are uniformly distributed. The Gemini-generated queries give every movie exactly 10 training and 10 test queries, creating a search dataset with no popularity bias. The authors acknowledge this explicitly: "the search popularity distribution is quite different from the recommendation distribution, and thus we might expect results to be more favorable in real-life distributions with some similarity between popularity distributions." This means the reported results may underestimate the benefit of Multi-task training, since real search and recommendation popularity are often correlated — popular items are both frequently searched and frequently interacted. However, the direction of this bias is only hypothesized, not tested with alternative query distributions.
No cold-start evaluation. The paper motivates Semantic IDs partly by their cold-start capability (items not seen during generative model training can still be represented if their embeddings fall into known codebook regions). However, no cold-start experiments are conducted. The uniform query distribution (every item has the same number of training queries) means that cold-start, in the sense of items with no search queries during training, does not exist in this dataset. The cold-start claim remains theoretical.
Small absolute performance levels. The best search R@30 is 0.072 and the best recommendation R@30 is 0.062. These are low recall figures — even the best model retrieves the correct item in the top 30 for only about 7% of search queries and 6% of recommendation prompts. The paper compares relative differences between methods at these low absolute levels. It is possible that the ranking of methods would change at higher absolute performance levels (achievable with larger generative models, more training data, or larger codebooks). The conclusions about Multi-task's relative advantage may not extrapolate.
Generative model scale is fixed. All experiments use Flan-T5-base (250M parameters). The relative benefits of shared vs. task-specific Semantic IDs may depend on model scale — larger models with more capacity might handle separate token spaces more effectively (by learning cross-space correlations in their larger parameter budgets) or might benefit even more from shared representations (by learning richer task-general item representations). The paper provides no evidence either way.

Missing Experiments

Several experiments would have strengthened the paper's conclusions or clarified open questions:

Single-task generative model with task-specific IDs as an upper bound. Train two separate Flan-T5-base models: one for search (using search-based IDs, trained only on search data) and one for recommendation (using recommendation-based IDs, trained only on recommendation data). Compare these single-task models' performance to the joint model with Multi-task IDs. This would establish whether the joint model's lower performance is due to the ID construction strategy or due to the inherent difficulty of multi-task generative training. If single-task models with task-specific IDs achieve substantially higher performance than the joint model with Multi-task IDs, the "effective trade-off" framing would need to acknowledge the cost of unification more explicitly.
Varying the balance of search vs. recommendation training data. The Multi-task bi-encoder is trained with the sum of search and recommendation contrastive losses, implying equal weight. The paper could have swept a weighting coefficient λ: L_total = L_search + λ × L_rec, where higher λ pushes the embeddings toward recommendation similarity and lower λ toward search similarity. This would produce a continuous Pareto frontier in Figure 1, showing whether practitioners can dial the trade-off to match their task priorities.
Matryoshka embeddings for natural dimensionality equalization. The paper mentions (in a footnote) that training embeddings with a Matryoshka objective would allow extracting both search and recommendation embeddings at equal dimensionality without SVD information loss. Running this experiment would test whether the FusedSVD result can be improved, and whether post-hoc fusion can approach Multi-task performance if dimensionality is handled properly. It would also clarify whether the remaining gap between FusedSVD (0.033/0.038) and Multi-task (0.046/0.049) is due to dimensionality effects alone or due to deeper representational incompatibility that only joint training can resolve.
Varying codebook size and number. The paper uses two codebooks of size 256 throughout. Ablating codebook count (1, 2, 3, 4) and codebook size (128, 256, 512) would reveal whether the trade-off between search and recommendation changes with representation capacity. With more tokens (e.g., 4 codebooks of 256), the quantization may preserve more information from both embedding spaces, potentially reducing the advantage of joint training over post-hoc fusion.
Cold-start item evaluation. Hold out a subset of items during generative model training and evaluate whether the model can generate their Semantic IDs at test time (when provided with search queries or recommendation contexts that should retrieve those items). This would empirically validate the cold-start claim that is currently only motivated theoretically.
Real search queries or alternative synthetic query distributions. The uniform query distribution is a known limitation. Evaluating with a skewed query distribution (popular movies get more queries, mimicking real search behavior) would test the authors' hypothesis that results would be "more favorable" under realistic distributions.

These missing experiments notwithstanding, the paper's core empirical contribution — the systematic comparison of Semantic ID construction strategies in a controlled joint search-and-recommendation setting — provides a solid foundation for its claims about relative method performance, even as it leaves several mechanistic questions and practical considerations open for follow-up work.

6. Limitations and Trade-offs

The Difficulty of Estimating Question Difficulty Is Unaccounted For

The assumption or constraint. The entire compute-optimal framework the paper advocates — adaptively allocating test-time inference compute based on prompt difficulty — rests on an ability to estimate question difficulty before deciding how to allocate the inference budget. The paper's method for doing so involves generating 2,048 samples per question, scoring them with the process reward model (PRM), and then binning questions into five difficulty quintiles based on the PRM's average final-answer score distribution. The authors acknowledge this cost explicitly in Section 3.2:

"We note that our method for predicting difficulty, which involves obtaining the 2048 samples and clustering the distribution of their average PRM probability, still incurs additional computation cost during inference. We do not account for the cost of computing difficulty in our experiments, largely for simplicity."

The consequence. This omission is severe. The difficulty estimation step requires 2,048 sample generations per prompt — more than the largest test-time compute budgets studied in the paper (256–512 generations). If this cost were amortized into the reported efficiency calculations, the claimed 4× advantage over best-of-N would shrink dramatically or potentially reverse at lower total budgets. For a prompt that receives a test-time budget of 64 generations, the true total cost including difficulty estimation is 2,048 + 64 = 2,112 generations — over 30× the nominal budget. At this total cost, best-of-N with 2,112 generations might well outperform the compute-optimal strategy with 64 generations plus difficulty estimation overhead. The paper does not report what accuracy a baseline achieves with the total budget of difficulty estimation plus strategy execution, so the true efficiency of the compute-optimal approach relative to a flat-budget baseline is unknown.

The issue is not merely an accounting technicality. It determines whether the compute-optimal framework is a practical deployment strategy or a theoretical analysis that requires oracle-level difficulty information. Without a cheap difficulty estimation method, the paper's results are best understood as an upper bound on achievable efficiency — what is possible if difficulty could be known for free — rather than a blueprint for deployment.

What evidence exists in the paper. The paper explicitly acknowledges this limitation in Section 3.2 and does not include the difficulty estimation cost in any budget calculation. The 2,048-sample protocol is described but its cost is never quantified relative to the evaluation budgets. Figure 4 and Figure 8, which report the compute-optimal scaling curves, plot accuracy against the strategy execution budget only (4, 16, 64, 256 generations), not the total budget including difficulty estimation. The predicted difficulty bins produce curves that "largely overlap" with oracle bins, which is encouraging for the feasibility of using PRM scores as a proxy for difficulty — but the cost of obtaining those PRM scores (the 2,048 generations plus scoring) is not factored into the curves.

Mitigation status. The paper acknowledges the limitation and frames it as a direction for future work, suggesting "pretraining or finetuning models to directly predict difficulty of a question" (Section 8). However, no such model is developed or evaluated. The paper also mentions in Section 3.2 that "We leave a more sophisticated treatment of the exploration–exploitation tradeoff involved in estimating difficulty as an area for future work," explicitly recognizing that the choice of how much compute to spend on difficulty estimation versus problem solving is itself an optimization problem. No solution or even partial approach is offered. The limitation is therefore entirely unresolved in the current work — the reported efficiency gains assume a solved difficulty estimation problem that remains unsolved.

Hard Problems Remain Unsolved Regardless of Compute Budget

The assumption or constraint. The paper's approach assumes that test-time compute can amplify the base model's existing capability, but it cannot create capability that is not there. This is not a hidden assumption — the authors are transparent about it — but it represents a fundamental bound on the approach's applicability. If the base model's pass@1 rate on a problem is near zero, no amount of search, revision, or adaptive allocation will surface a correct answer, because there are no correct solutions in the proposal distribution to find or refine.

The consequence. Across all methods studied — search, revisions, and their compute-optimal combinations — the hardest questions (difficulty bin 5) show essentially zero improvement regardless of compute budget. In the PRM search experiments (Figure 3, right), bin 5 accuracy hovers at 1–3% for all methods and all budget levels from 4 to 256 generations. In the revision experiments (Figure 7, right), bin 5 accuracy is roughly 2–3% regardless of the sequential-to-parallel ratio. In the FLOPs-matched comparison (Figure 9), the bin 5 scaling lines are essentially flat near 0–5%, well below the ~14× larger model's greedy performance (indicated by the orange star). The compute-optimal policy cannot help here because none of the candidate strategies work — best-of-N fails, beam search fails, revisions fail, and adaptive allocation cannot select a working strategy where none exists.

This means the approach offers no path forward for genuinely novel or out-of-distribution reasoning that exceeds the base model's training distribution. For problems where the model has no latent capability — it does not produce correct solutions at any non-trivial rate — test-time compute is useless. Pretraining remains the only viable path for expanding the frontier of solvable problems. The paper's FLOPs-matched analysis (Section 7) makes this concrete: on hard questions at high inference-to-pretraining ratios (R ≫ 1), test-time compute shows a −52.9% relative disadvantage compared to the larger pretrained model for PRM search, and a −37.2% relative disadvantage for revisions (bar charts in Figure 1). The paper frames this as a clean boundary condition — test-time compute can substitute for pretraining only within the base model's capability envelope — but it means the technique is not a general-purpose solution for improving LLM reasoning. It helps only on problems the model already can solve, just not reliably.

What evidence exists in the paper. The failure is documented across every experiment that breaks out performance by difficulty: Figure 3 (right, bin 5), Figure 7 (right, bin 5), Figure 9 (bin 5 lines vs. orange stars). The paper's Section 7 takeaway box explicitly states that "Improvements were most significant for easy and medium questions, with minimal gains observed for hard questions." The evidence is consistent and unambiguous.

Mitigation status. The paper is transparent about this limitation — it does not claim to solve hard problems. The difficulty-binned analysis is presented because the authors want to show where the approach works and where it does not. However, no mitigation is attempted. The limitation is inherent to the approach: test-time compute can only work with what the base model can produce. Future work on improving the base model's capability on hard problems (through better pretraining, retrieval augmentation, or fundamentally different reasoning architectures) is orthogonal to the test-time compute allocation framework the paper studies.

The ~14× Larger Pretrained Model Baseline Is Not Compute-Optimally Trained

The assumption or constraint. The FLOPs-matched comparison in Section 7 compares PaLM 2-S* augmented with compute-optimal test-time compute against a model with approximately 14× more parameters, trained with the same data but scaling only parameters (not data volume). The authors explicitly state (Section 7):

"We choose this setting as it is representative of a canonical approach to scaling pretraining compute (e.g. LLaMA [Touvron et al., 2023]) and leave the analysis of compute-optimal scaling of pretraining compute where the data and parameters are both scaled equally (e.g. Chinchilla [Hoffmann et al., 2022]) to future work."

The consequence. This design choice biases the comparison in favor of test-time compute. Hoffmann et al. (2022) established that compute-optimal pretraining scales model parameters and training tokens approximately equally — doubling total FLOPs should increase both parameters and data by roughly √2×. The Chinchilla scaling laws suggest that a model trained with only parameter scaling (fixed data) is undertrained relative to its parameter count and underperforms a compute-optimally trained model at the same total FLOPs budget. The larger model in the paper's comparison — a parameter-scaled LLaMA-style model — is therefore likely weaker than a properly compute-optimal model at the same FLOPs would be. If the baseline were trained compute-optimally (scaling both parameters and data), its performance advantage over the small model would be larger, and the regimes where test-time compute "outperforms" it would shrink.

Additionally, the larger model uses only greedy decoding — no majority voting, no best-of-N, no beam search. Giving the larger model even a modest test-time compute budget (e.g., best-of-8 or best-of-16) would create a much stronger baseline that aligns more closely with how large models are deployed in practice. The current comparison is essentially "small model with smart inference vs. large model with naive inference," which conflates the value of test-time compute with the value of smarter inference strategies in general. The fair comparison — "small model with smart inference vs. large model with smart inference, under equal total FLOPs" — is not presented.

The quantitative impact of this baseline choice is unknown. It is possible that against a compute-optimally trained larger model with its own test-time compute budget, the small model's advantage on easy-to-medium problems narrows or disappears. The paper's claim that "test-time compute can outperform a 14× larger model" is therefore conditional on the larger model being suboptimally trained and deployed — a caveat that should qualify the strength of the conclusion.

What evidence exists in the paper. The paper provides no ablation varying the pretraining compute allocation of the larger model. The LLaMA-style scaling is stated as a deliberate design choice, and the Chinchilla-optimal alternative is acknowledged as future work. The FLOP accounting formula (Section 7) assumes scaling parameters by M multiplies both pretraining FLOPs and inference FLOPs by M, but this accounting does not model the performance consequences of how those pretraining FLOPs are allocated between parameters and data. The experimental results (Figure 9, bar charts in Figure 1) therefore reflect the specific baseline chosen, and their generalizability to compute-optimal pretraining is untested.

Mitigation status. The authors explicitly flag this as a limitation (Section 7) and defer it to future work. No sensitivity analysis or partial exploration (e.g., testing different data-to-parameter ratios for the larger model) is conducted. The mitigation is entirely in the acknowledgment, not in the experimental design.

Single Benchmark, Single Model Family, No Evidence of Cross-Domain Generalization

The assumption or constraint. All experiments in the paper use the MATH benchmark (Hendrycks et al., 2021) — specifically, the 500-question test split from Lightman et al. (2022) — with PaLM 2-S* (Anil et al., 2023) as the base model. The authors state in Section 4 that they "believe this model is representative of the capabilities of many contemporary LLMs," but no evidence is provided to support this representativeness claim. The findings about difficulty-dependent optimal strategies, the $4 \times$ efficiency gain from compute-optimal allocation, and the FLOPs-matched comparison against a 14× larger model are all conditional on this specific model–benchmark pair.

The consequence. Several aspects of the paper's findings could be model-specific or domain-specific in ways that affect their practical applicability:

PRM over-optimization behavior depends on verifier quality. The finding that beam search degrades performance on easy problems at high budgets (Figure 3, right) is a function of the specific PRM trained with Monte Carlo rollouts from PaLM 2-S*. A PRM trained on a different model's outputs (with different calibration, different error patterns, different step-level score distributions) could exhibit different over-optimization thresholds, changing which strategies are optimal at which difficulty levels. The compute-optimal policy — which selects best-of-N for easy problems and beam search for medium problems — is specifically tuned to this verifier's reliability profile.
Revision model effectiveness depends on the base model's in-context learning ability. The revision model is fine-tuned to condition on its own previous incorrect answers and produce improved answers. This skill depends on the base model's capacity for in-context learning and self-correction, which varies substantially across model families. PaLM 2-S* may be better or worse at this than GPT-4, Claude, LLaMA, or other models, and the ~25% final-step pass@1 achieved after 15–20 revisions (Figure 6, left) may not generalize.
The MATH benchmark consists of competition-level math problems requiring symbolic reasoning and specific mathematical knowledge. The difficulty-dependent patterns — beam search helping medium problems but hurting easy ones, revisions helping easy problems but requiring parallel sampling for hard ones — may be specific to mathematical reasoning where problems have clear step-by-step structure that the PRM can evaluate. For other reasoning domains (code generation, logical reasoning, scientific QA, multi-hop question answering) or for tasks requiring factual recall rather than inference, the optimal strategy profile could be entirely different. The paper's difficulty estimation method (pass@1 on 2,048 samples) is feasible on MATH because answers can be automatically graded — extending this to open-ended generation tasks would require fundamentally different verifier and difficulty estimation approaches.
The test set is 500 questions. After partitioning into five difficulty quintiles (~100 questions each) and applying two-fold cross-validation (halving each bin for strategy selection vs. evaluation), the compute-optimal policy is selected based on approximately 50 questions per bin per fold. At this sample size, the selected strategies may be noisy — a different random split or a different set of 500 questions could yield different optimal strategies per bin. The paper does not report confidence intervals for the compute-optimal scaling curves (Figures 4 and 8), making it impossible to assess whether the observed improvements are statistically reliable or could be explained by variance in strategy selection on small samples.

The consequence is that a practitioner cannot confidently apply the paper's specific recommendations — "use best-of-N on easy problems, beam search on medium problems, revisions on easy problems, balanced sequential-parallel on hard problems" — to a different model or domain without conducting their own full-scale replication of the analysis. The framework (difficulty-conditioned compute-optimal allocation) may generalize, but the specific policies derived from this model–benchmark pair likely do not.

What evidence exists in the paper. There is none. The paper does not report experiments on any benchmark other than MATH, any base model other than PaLM 2-S*, or any domain other than mathematical reasoning. The cross-validation protocol is described in Section 3.2 but no variance estimates for the compute-optimal curves are provided. The claim that PaLM 2-S* is "representative" is stated without supporting evidence or argument.

Mitigation status. Not addressed. The paper does not frame the single-benchmark, single-model scope as a limitation requiring future work. The "representative" claim is presented as a justification for the model choice rather than as a hypothesis to be tested. Section 8's future work suggestions focus on combining search with revisions, improving difficulty estimation, and self-improvement loops — not on cross-model or cross-domain replication. This is a significant gap given that the paper's practical value to practitioners depends on the transferability of its findings.

The Revision Model Has a 38% Correct-to-Incorrect Reversion Rate and Training Is Brittle

The assumption or constraint. The revision model is trained using offline data construction (Section 6.1): for each training question, the authors sample independently generated correct and incorrect solutions from the base model, pair them using edit distance as a proxy for trajectory coherence, and fine-tune the base model to produce the correct answer given a sequence of incorrect answers in context. Because the training data only contains incorrect-to-correct transitions, the model learns to always produce a different answer when conditioned on previous answers — it has no training signal for recognizing when the current answer is already correct and should be preserved.

The consequence. At inference time, approximately 38% of correct answers produced during a revision chain get "revised" back into incorrect answers in the subsequent revision step (Section 6.1). This is a direct and predictable consequence of the training data construction: the model's training distribution never includes examples where the in-context answer is correct, so at test time, when the model encounters its own correct output from a previous revision, it has no learned behavior for "leave it alone." The paper mitigates this with selection mechanisms — majority voting or verifier-based selection across the entire revision chain, picking the best answer from any point rather than trusting the final revision — but these are patches that treat the symptom rather than the cause. The underlying model has a learned tendency to alter correct answers, which means the revision chain is actively destructive approximately 38% of the time a correct answer is reached.

Beyond the reversion problem, the revision training procedure is demonstrably brittle. The ReST $^{EM}$ experiment (Appendix K, Figure 16) shows that attempting to further optimize the revision model with reinforcement learning (ReST $^{EM}$ ; Singh et al., 2024) causes performance to degrade substantially: at 256 generations, fully sequential revisions with the ReST $^{EM}$ -trained model drop to approximately 33.5% accuracy compared to roughly 38.5% at the optimal ratio for the base revision model. The authors hypothesize that "the on-policy data collection in ReST $^{EM}$ exacerbates spurious correlations in revision data, causing the model to fail to learn the revision task properly." This indicates that the revision approach is sensitive to training methodology — what works with offline, edit-distance-paired data construction may not survive on-policy optimization — and that the positive revision results depend on specific design choices that may not transfer to other settings.

For a practitioner, this brittleness is a significant concern. It means the revision model cannot be straightforwardly improved through iterative self-play or RL-based optimization, which are standard techniques for improving model outputs in other domains. The revision model is effectively frozen at the quality level achieved by the initial supervised fine-tuning on offline data, with no clear path for further improvement without risking the ReST $^{EM}$ degradation.

What evidence exists in the paper. The 38% reversion rate is reported in Section 6.1. The ReST $^{EM}$ degradation is documented in Appendix K and Figure 16. The paper's mitigation — within-chain selection via majority voting or verifier — is evaluated in Figures 6, 7, and 8, showing that sequential revision with selection still outperforms parallel sampling, but the gap is narrower than it would be without the reversion problem. The paper does not report what the revision performance would be if the model could reliably preserve correct answers (e.g., by training on trajectories that include correct-in-context examples with a "no change" target).

Mitigation status. Partially mitigated through selection mechanisms (majority voting, verifier-based best-of-N weighted across the chain), which recover the best answer from any point in the revision chain rather than relying on the final output. However, this mitigation does not address the root cause — the model remains prone to revising correct answers into incorrect ones — and it adds computational overhead (the verifier must score every step of the chain). A more principled solution, such as training the model with mixed trajectories that include correct-in-context examples and a special "no revision needed" token, is not explored. The ReST $^{EM}$ failure is noted as an empirical observation but no solution is proposed; the authors simply report it as a negative result and do not pursue it further.

Sequential Revision Strategies Are Inherently Latency-Bound

The assumption or constraint. The paper measures test-time compute in "generations" — the number of complete solutions sampled from the model — which serves as a reasonable proxy for total FLOPs. However, this metric ignores latency (wall-clock time). Sequential revisions are inherently serial: each revision depends on the previous one, so generating a chain of 64 sequential revisions requires 64 sequential forward passes through the model, even if the total FLOPs are comparable to 64 parallel samples (which could be executed simultaneously with sufficient hardware).

The consequence. A strategy that the compute-optimal policy selects for easy problems — pure sequential revision, which Figure 7 (left and right) shows performs best or near-best on bins 1–2 — consumes the same total FLOPs as 64 parallel samples but takes approximately 64× longer in wall-clock time on hardware with sufficient parallelism to run the parallel samples concurrently. For latency-sensitive applications (interactive assistants, real-time decision-making, user-facing chatbots), this renders sequential-heavy strategies impractical regardless of their accuracy advantages. A user waiting for a math answer will not tolerate a 60× increase in response time for a few percentage points of accuracy gain.

The paper's compute-optimal policy selects strategies based solely on accuracy per generation FLOP, without any latency constraint. In practice, deployment systems have latency budgets — "respond within 500ms" or "respond within 2 seconds" — and strategies that exceed this budget are infeasible regardless of their FLOP efficiency. The paper provides no guidance for how to incorporate latency constraints into the compute-optimal allocation framework. A practitioner reading the paper would learn that sequential revisions work well on easy problems but would not know whether the 4× efficiency gain is realizable under realistic latency constraints.

Furthermore, the trade-off between latency and throughput creates a tension the paper does not address. Parallel best-of-N can utilize hardware efficiently by batching samples and achieving high throughput. Sequential revisions tie up hardware resources for extended periods on single prompts, reducing overall system throughput even if per-prompt FLOP efficiency is high. For batch processing (evaluating many prompts offline), throughput matters more than per-prompt latency, and the sequential strategies favored by the compute-optimal policy may underperform when measured in completions-per-hour rather than accuracy-per-FLOP.

What evidence exists in the paper. None. The paper does not discuss latency, wall-clock time, or throughput. All budgets are measured in generations. The hardware configuration (number of GPUs, whether parallel samples are batched or executed concurrently, whether model parallelism is used) is not specified. There is no latency-constrained allocation experiment, no Pareto frontier showing the accuracy–latency trade-off for different strategies, and no discussion of how latency considerations would change the optimal policy.

Mitigation status. Not addressed at all. The paper frames test-time compute allocation purely as a FLOPs optimization problem, ignoring the temporal dimension entirely. In the deployment contexts the paper motivates — on-device deployment (Section 1), self-improvement pipelines (Section 8), and inference-time scaling to compete with larger models — latency is a first-order constraint that would significantly reshape the optimal allocation policy. The omission is understandable as a scope limitation (the paper is long and already covers substantial ground), but it means the reported strategy recommendations are incomplete for any practitioner operating under real-world latency budgets.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper shifts the conversation around item representations in generative retrieval from a single-task optimization mindset to a multi-task representation learning problem. Prior to this work, the dominant approach in generative retrieval research — visible in DSI (Tay et al., 2022), TIGER (Rajput et al., 2023), RIPOR (Zeng et al., 2024), and TokenRec (Qu et al., 2024) — was to optimize item embeddings for a specific task, quantize them into Semantic IDs, and then use those IDs in a generative model. The implicit assumption was that task-specialized embeddings produce the best task-specialized Semantic IDs. This paper demonstrates that when multiple retrieval tasks share a single generative model, that assumption leads to a representation collision: Semantic IDs optimized for one task systematically degrade performance on the other, and post-hoc attempts to fuse separately trained embeddings cannot fully recover the lost performance.

The paper's core finding reframes the problem: the representation tension between search and recommendation is not an irreducible conflict that requires task-specific ID spaces to resolve, but rather a training artifact that joint embedding optimization can substantially mitigate. Multi-task embedding training — where a single bi-encoder is trained with the sum of search and recommendation contrastive losses before quantization — produces Semantic IDs that achieve search R@30 of 0.046 and recommendation R@30 of 0.049 (Table 2), competitive with both task-specific baselines (0.072 search, 0.062 recommendation) without any task-specific tokens or post-hoc fusion. This challenges the prevailing intuition — visible in the concurrent Prefix-share work by Shi et al. (2025) — that some degree of task-specific representation capacity is necessary for multi-task generative retrieval. The paper provides evidence that it is not only unnecessary but potentially harmful, since token separation (Separate, search R@30 = 0.028, recommendation R@30 = 0.032) prevents cross-task knowledge transfer through shared token embeddings.

This is an architectural reframing rather than a paradigm shift. The paper does not introduce a new model class, a new training algorithm, or a new theoretical framework. It takes a design choice that every joint generative model builder must make — how to construct item embeddings before quantization — and systematically characterizes the performance consequences of different approaches, producing a clear recommendation (train embeddings jointly on all task objectives) and a clear anti-pattern (do not give each task separate ID tokens). The contribution is in the systematic comparison, not in the novelty of any individual method.

The paper also provides a reconciliation of conflicting intuitions in the broader multi-task learning literature. One line of thought — common in NLP — holds that task-specific output heads prevent interference and enable specialization. Another line — from multi-domain recommendation — holds that shared representations enable beneficial cross-domain transfer. The paper's results resolve this for the specific case of generative retrieval: shared item representations (through common Semantic ID tokens) outperform task-separated representations when tasks operate on the same item catalog, but only if the embeddings that produce those tokens are themselves learned jointly rather than separately optimized and then composed. The Separate approach — which is the natural implementation of the "task-specific output heads" intuition — fails because it prevents knowledge transfer between tasks in the generative model. The Fused approaches — which implement the "shared representation via post-hoc composition" intuition — fail because independently trained embeddings inhabit incompatible subspaces that do not compose cleanly through concatenation or summation. Multi-task succeeds because it resolves the tension at the embedding learning stage itself, before quantization or generative model training.

Several research directions become more attractive in light of these findings:

Joint embedding training for multi-task retrieval is now the default recommendation, displacing the per-task optimization followed by ad-hoc fusion that characterized prior work. This shifts research attention from "how to combine task-specific embeddings" to "how to train embedding models on diverse, potentially conflicting supervision signals."
Simple quantization (RQ-KMeans) is validated as a strong baseline that may not need the complexity of learned auto-encoder approaches (RQ-VAE, ResidualLFQ) when the input embeddings are already well-structured by contrastive or supervised objectives. Table 3 provides direct evidence: RQ-KMeans achieves search R@30 = 0.046 vs. RQ-VAE's 0.002 on the same Multi-task embeddings. This reorients quantization research from architectural sophistication toward robustness and stability.
Cross-task regularization through shared tokens emerges as a mechanism worth studying in its own right, rather than an incidental property of vocabulary sharing. The paper's explanation for Separate's failure — that disjoint task vocabularies prevent knowledge transfer — is plausible but not mechanistically demonstrated, opening questions about exactly how shared token embeddings facilitate multi-task learning in generative models.

Research directions that become less attractive — or at least require stronger justification — include:

Auto-encoder-based quantization (RQ-VAE) as the default for Semantic ID construction. The paper shows it can catastrophically underperform RQ-KMeans (search R@30 of 0.002 vs. 0.046 on Multi-task embeddings), consistent with Hong et al. (2025) similarly abandoning it for hierarchical k-means. Future work using RQ-VAE must demonstrate it beats RQ-KMeans, not merely assume it.
Task-specific ID spaces as a design pattern for multi-task generative retrieval. Separate's failure (search 0.028, recommendation 0.032 vs. Multi-task's 0.046/0.049) and Prefix-share's underperformance (0.007/0.021) provide concrete evidence against this approach, at least at the scale studied and with current quantization methods.
Post-hoc embedding fusion without joint training. The gap between Multi-task (0.046/0.049) and FusedSVD (0.033/0.038) — despite FusedSVD's dimensionality equalization — suggests that independently trained embeddings contain representational incompatibilities that no post-hoc composition can fully resolve. Future systems should invest in joint embedding training rather than in more sophisticated fusion strategies.

Follow-Up Research This Work Enables

Matryoshka embedding training for natural dimensionality equalization in multi-task fusion. The paper identifies dimensionality imbalance as a concrete failure mode in Fusedconcat (the 768-dimensional search embedding dominates the 256-dimensional recommendation embedding, producing search R@30 = 0.048 but recommendation R@30 = 0.018). FusedSVD partially addresses this by projecting the search embedding down to 256 dimensions via truncated SVD, but this discards information and still underperforms Multi-task (0.033/0.038 vs. 0.046/0.049). The paper mentions in a footnote that training embeddings with a Matryoshka objective (Kusupati et al., 2022) — where the model learns to produce useful representations at multiple dimensionalities — would allow extracting both search and recommendation embeddings at equal dimensionality without information loss. A direct follow-up would train a bi-encoder with a Matryoshka loss on the MovieLens25M search–recommendation setup, extract 256-dimensional search and recommendation embeddings, apply FusedSVD-style summation (or an alternative such as learned weighted summation), quantize with RQ-KMeans, and evaluate in the same joint Flan-T5-base generative model. The key comparison is whether Matryoshka-based fusion approaches or matches Multi-task performance (0.046/0.049). A positive result — Matryoshka fusion ≈ Multi-task — would suggest that post-hoc fusion can work if embeddings are trained to be compatible at the same dimensionality from the start, narrowing the practical advantage of joint training. A negative result — Matryoshka fusion still underperforms Multi-task — would strengthen the paper's conclusion that joint training provides benefits beyond dimensionality compatibility (e.g., shared gradient signals push embeddings into subspaces that are inherently more composable).

Scaling the generative model size to test whether Multi-task's advantage holds or narrows. All experiments in the paper use Flan-T5-base (~250M parameters). The advantage of shared Semantic IDs (Multi-task) over token-separated IDs (Separate) is attributed to cross-task regularization through shared token embeddings — during joint training, the generative model learns task-general item representations because the same output tokens are used for both search and recommendation. This mechanism may depend on model scale: smaller models with limited capacity may benefit more from shared representations because they cannot afford to learn separate, high-quality embeddings for disjoint task vocabularies. Larger models with more parameters might handle token separation more effectively by learning cross-task correlations in their larger parameter budgets, narrowing the gap between Separate and Multi-task. A follow-up would replicate the Separate vs. Multi-task comparison with Flan-T5-large (770M), Flan-T5-XL (3B), and (if resources permit) Flan-T5-XXL (11B), keeping the MovieLens25M dataset, the same embedding models, and the same training regime (3 epochs, batch 128, learning rate 0.002). The prediction from the paper's logic: the Separate-vs-Multi-task gap should persist or widen at larger scales because shared tokens are inherently more parameter-efficient and provide a structural inductive bias toward cross-task generalization that larger capacity alone cannot replicate. A narrowing gap would suggest the Separate strategy is viable at sufficient scale, undermining the paper's strongest architectural recommendation.

Cross-domain evaluation with real (non-synthetic) search queries and a different item catalog. The paper's experiments use MovieLens25M with synthetically generated search queries (10 training + 10 test queries per movie, generated by Gemini-2.0-flash). The authors explicitly note that this uniform query distribution "is quite different from the recommendation distribution, and thus we might expect results to be more favorable in real-life distributions with some similarity between popularity distributions." A direct follow-up would replicate the embedding construction comparison (Search-based, Recommendation-based, Fusedconcat, FusedSVD, Multi-task, Separate) on a dataset with real search queries and natural popularity distributions. Candidates include Amazon review data (product search queries available in some Amazon dataset variants), an e-commerce dataset with clickstream logs, or a music streaming dataset with real user queries. The key measurements: (1) whether Multi-task's advantage over task-specific baselines widens when search and recommendation popularity are correlated (as the authors hypothesize), and (2) whether the ranking of methods changes in a domain where content and collaborative signals are more orthogonal than in movies (e.g., fashion e-commerce, where purchase co-occurrence may have little to do with product description similarity). A finding that Multi-task works across domains would make the paper's recommendation general; a finding that it is domain-specific would establish important boundary conditions.

Cold-start evaluation to empirically validate the Semantic ID generalization claim. The paper motivates Semantic IDs partly by their ability to represent unseen items: "items with similar content (embeddings) share tokens, improving generalization and enabling cold start settings." However, no cold-start experiment is conducted — all items in MovieLens25M appear during generative model training. A follow-up would hold out a subset of movies (e.g., the 20% least popular by training interactions) during generative model training, then evaluate whether the trained model can successfully generate the Semantic IDs of these held-out items when provided with search queries or recommendation contexts that should retrieve them. The experiment would compare Multi-task embeddings against content-based embeddings (which should perform best for cold start since they rely purely on metadata similarity) and recommendation-based embeddings (which should fail for items unseen during ENMF training). This would quantify the cold-start trade-off that is currently only claimed qualitatively: does Multi-task's balanced search–recommendation performance come at the cost of reduced generalization to unseen items compared to pure content-based approaches?

Gradient-based analysis of cross-task regularization in shared vs. separate token embeddings. The paper's explanation for Separate's underperformance — "the knowledge learned from one task cannot be used for the task-specific tokens of the other task, negating the regularization effect in item representations" — is an interpretation, not a demonstrated mechanism. A follow-up would instrument the generative model training to directly measure whether shared token embeddings enable cross-task transfer. Concretely: during joint training of the Flan-T5-base model with shared Multi-task IDs, measure the cosine similarity between the decoder's token embeddings for items that frequently co-occur in both search results and recommendation contexts (high cross-task overlap) vs. items that are only retrieved for one task. Compare this to the Separate model, where each item has two disjoint token embeddings (one for search, one for recommendation). If the paper's regularization hypothesis is correct, the shared-ID model should learn more similar representations for search and recommendation facets of cross-task-overlapping items than the separate-ID model, and this similarity should correlate with downstream task performance. A negative result — shared IDs do not produce more similar cross-task representations — would force a reevaluation of the mechanism underlying Multi-task's advantage, pointing instead to parameter efficiency or vocabulary-size effects.

Practical Applications and Downstream Use Cases

Unified product search and recommendation on e-commerce platforms. A large e-commerce platform (e.g., Amazon, Etsy, Shopify stores) currently maintains separate models for product search (retrieving relevant products given a user query) and product recommendation (predicting the next product a user will interact with given their browsing or purchase history). The paper's findings suggest that a single Flan-T5-base (or larger) generative model using Multi-task Semantic IDs — constructed by fine-tuning a product embedding model jointly on query–product relevance data and product co-occurrence data, then quantizing with RQ-KMeans — can serve both tasks at competitive quality. The concrete benefit: engineering consolidation (one model serving stack, one embedding pipeline, one index of Semantic IDs) without sacrificing recommendation quality (Multi-task recommendation R@30 = 0.049 is within 21% of the recommendation-only upper bound of 0.062) and with manageable search degradation (Multi-task search R@30 = 0.046 is within 36% of the search-only upper bound of 0.072). For platforms where search is the dominant interaction mode, the trade-off may be acceptable; where a slight drop in per-task performance is offset by reduced operational complexity and faster iteration cycles across unified infrastructure. The token budget efficiency (512 tokens for 62K items, or roughly 0.008 tokens per item) means the approach scales to catalogs of millions of products without exploding vocabulary size. The cold-start property — new products can be represented immediately if their embedding falls into existing codebook regions — enables real-time catalog updates without model retraining, which is critical for platforms with rapidly changing inventory.

Joint content discovery in media streaming services. A music or video streaming service (e.g., Spotify, Netflix, YouTube) operates both search (users typing "upbeat workout playlist" or "Oscar-winning dramas") and recommendation (next-track prediction, personalized homepage rows). Currently, the item representations used by the search ranking stack (typically content-based embeddings from audio features, metadata, and lyrics) are distinct from those used by the recommendation stack (typically collaborative filtering embeddings from listening/watch history). The paper's Multi-task approach suggests a unified pipeline: train a bi-encoder on both query–track similarity and track co-occurrence (from user listening sessions), quantize into Semantic IDs, and serve both search and recommendation from a single generative model. The specific benefit: content-based embeddings (Table 1, row 1: search R@30 = 0.013, recommendation R@30 = 0.023) underperform task-tuned alternatives on both tasks, meaning a system relying on pure content-based Semantic IDs leaves substantial performance on the table. Recommendation-tuned embeddings (search 0.004) fail catastrophically at search, making them unusable for a joint system. Multi-task (search 0.046, recommendation 0.049) provides a single set of Semantic IDs that work for both, without the 2× vocabulary expansion and isolation problems of Separate (0.028/0.032). For a streaming service with hundreds of millions of tracks, the token budget remains manageable (512 tokens for the codebooks, independent of catalog size), and the RQ-KMeans-based quantization can be run efficiently on large embedding sets using FAISS (Douze et al., 2024). The popularity-stratified results in Table 2 provide guidance: Multi-task performs well on Head items (recommendation R@30 = 0.135, close to the recommendation-only 0.170) but weaker on Torso items (0.024 vs. recommendation-only 0.035). This suggests the unified approach is most appropriate for services with heavy-tailed consumption patterns where Head items dominate, and it may need supplementation (e.g., separate retrieval for long-tail items) in services where Torso coverage is critical.

Generative retrieval in domain-specific enterprise search and recommendation. An enterprise knowledge management system — e.g., internal document search plus "related documents" recommendation for legal, medical, or technical corpora — can use the Multi-task Semantic ID approach to build a single retrieval model over a specialized document catalog. The advantage over traditional dual-encoder retrieval is that the generative model can be fine-tuned end-to-end on the specific corpus and task distribution, learning to map domain-specific queries and interaction patterns to document IDs. The paper's findings imply that the embedding model should be fine-tuned jointly on both the enterprise's query–document pairs (derived from search logs) and document–document co-occurrence (derived from user access patterns or citation graphs) rather than using a generic pre-trained text embedder. A concrete deployment: fine-tune all-mpnet-base-v2 on the enterprise's own (query, document) pairs and (document, co-accessed-document) pairs, quantize with RQ-KMeans into Semantic IDs, train Flan-T5-base jointly on search and recommendation prompts, and deploy with diversified beam search for retrieval. The ~250M parameter generative model can run on a single GPU for moderate-scale corpora (tens of thousands of documents), making it feasible for on-premises deployment. The paper's quantization ablation (Table 3) directly informs the tokenizer choice: RQ-KMeans, not RQ-VAE, should be the default, saving engineering effort on hyperparameter tuning for unstable auto-encoder quantizers.

When to Prefer This Method Over Alternatives

The paper does not articulate a structured decision rule for choosing among specific alternative architectures or frameworks, and its experimental scope is limited to one dataset (MovieLens25M), one generative model (Flan-T5-base), and two tasks (search and recommendation). The findings, however, imply practical trade-offs that can be extracted for the specific setting studied:

Prefer Multi-task Semantic IDs when a single generative model must serve both search and recommendation over the same item catalog, the token budget is constrained (you want ~512 new vocabulary entries rather than 1,024), and you can afford to jointly train an embedding model on both task objectives before quantization. This provides the best balanced performance (search R@30 = 0.046, recommendation R@30 = 0.049) at minimal token cost.
Prefer Search-based Semantic IDs when search performance dominates your loss function, recommendation is a secondary concern, and you are willing to accept recommendation R@30 of 0.026 (a 58% reduction from the recommendation-specific baseline) in exchange for the best search performance (0.072). This is the appropriate choice for a primarily search-oriented deployment where recommendation is a nice-to-have.
Prefer Recommendation-based Semantic IDs when the reverse holds — recommendation quality is critical and search is rarely used or can be handled by a separate system. Recommendation R@30 of 0.062 is the upper bound in this setup, though search collapses to near zero (0.004).
Avoid Separate and Prefix-share at the scale studied: they underperform Multi-task on all metrics while increasing token budget, model complexity, or reliance on underperforming quantization methods. Separate's double vocabulary (1,024 tokens) and Prefix-share's auto-encoder requirement yield no benefit over simpler, embedding-level combination.
Use RQ-KMeans, not RQ-VAE, for quantization based on the ablation in Table 3: RQ-KMeans achieves search R@30 of 0.046 vs. RQ-VAE's 0.002 on the same Multi-task embeddings. The gap is large enough that RQ-VAE should not be the default without strong evidence of advantage in a specific setting.

These preferences are conditional on the MovieLens25M setup with synthetic queries and Flan-T5-base. They should not be treated as universal without replication on other datasets, models, and domains. The paper does not provide evidence that these preferences generalize; it provides evidence that they hold for the specific configuration studied.