Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge

🎯 Pitch

Instead of committing to one next word, this work lets LLMs privately sample multiple candidates at each step and blend them into a single 'multiplex token,' creating a soft, BFS-like superposition over reasoning paths that does not lengthen the sequence. This simple mechanism dramatically expands exploration capacity, enabling RL training on the model’s own token-level uncertainty and yielding major Pass@k gains—over 15 points absolute—on competition math benchmarks while actually shortening generations.

1. Executive Summary

This paper introduces Multiplex Thinking, a stochastic soft reasoning mechanism that, at each thinking step, independently samples K candidate tokens and aggregates their embeddings into a single continuous multiplex token (e.g., averaging 3 sampled token embeddings instead of committing to one discrete token). Evaluated on DeepSeek-R1-Distill-Qwen-1.5B and 7B backbones across six math reasoning benchmarks (AIME 2024/2025, AMC 2023, MATH-500, Minerva, OlympiadBench), multiplex thinking consistently outperforms discrete CoT, discrete RL, and Stochastic Soft Thinking baselines—achieving the best Pass@1 in 11 of 12 settings and a widening Pass@k advantage up to approximately 55% vs. 40% on AIME 2025 (7B) at k=1024—while producing shorter sequences by compressing high-entropy reasoning steps into information-dense tokens. The gains hold both with and without RL fine-tuning, establishing that multiplex representations intrinsically expand exploration capacity, though marginal returns diminish beyond moderate widths (K=2–3) and the hardest benchmarks still reveal performance ceilings tied to base model capability.

2. Context and Motivation

The Core Problem: Reasoning with Discrete Tokens Is Expensive and Explores Poorly

The fundamental tension this paper addresses is straightforward but deeply consequential: chain-of-thought reasoning, the dominant paradigm for eliciting complex problem-solving from LLMs, forces the model to commit to a single discrete token at every step. When a model generates a CoT trace, it samples one token from its vocabulary at a time—"the," then "car," then "different," and so on. Each of these discrete choices irreversibly discards the rich probability distribution the model computed over its entire vocabulary. The model knew, at that moment, that "first" and "different" were both plausible continuations with non-trivial mass, but it had to pick one and abandon the other. All the information encoded in the relative probabilities of alternatives—the model's uncertainty, the branching structure of possible reasoning paths—vanishes into a single one-hot vector.

This is not merely an aesthetic concern about information loss. It has direct consequences for both the cost and the exploration quality of reasoning. Each reasoning path is a long sequence of discrete tokens, and exploring alternatives requires generating entirely separate rollouts—essentially a depth-first search through the space of possible thoughts. As the paper notes, this means test-time compute scales roughly linearly with the number of paths explored: self-consistency with 64 samples costs 64× the compute of a single greedy decode, best-of-N selection with an outcome reward model requires N full generations, and so on. The bandwidth of reasoning—how much thinking occurs per token—is low because each token carries exactly one bit of vocabulary choice.

Why This Matters: The Real-World and Theoretical Stakes

The paper is motivated by two converging pressures on LLM deployment:

Economic pressure: the cost of long reasoning traces. As models like OpenAI's o1 and DeepSeek-R1 demonstrate, scaling test-time compute through extended chains of thought produces substantial accuracy gains on hard problems. But these gains come at a steep price: users pay per output token, and long CoT traces can easily consume thousands of tokens before producing a final answer. A multiplex token that encodes multiple plausible next steps in a single vector—rather than serializing them across a long sequence of discrete deliberation—offers the possibility of achieving similar reasoning depth with shorter, cheaper outputs. The paper's finding that multiplex thinking produces meaningfully shorter sequences than discrete baselines while achieving higher accuracy (Figure 5, Table 3) makes this economic case directly.

Exploration pressure: discrete sampling limits the search space. A more subtle but equally important problem concerns how RL-trained reasoning models explore during training. In standard RL with verifiable rewards (RLVR, described in Section 2), the model generates complete CoT rollouts, receives a reward signal based on the final answer's correctness, and reinforces behaviors that led to correct answers. But because each rollout commits to a single discrete trajectory, the model's exploration is depth-first: it samples one complete reasoning path from start to finish, evaluates it, and adjusts. Paths that start identically but diverge at a later critical decision point require separate rollouts to discover. Multiplex thinking, by contrast, operates in a fundamentally different exploration mode: at each step, it simultaneously samples K independent tokens and aggregates them, meaning a single multiplex trajectory implicitly explores a "superposition" of $K^L$ possible discrete paths (where $L$ is the number of thinking steps) while only paying the cost of one sequence of length $L$ . This transforms the exploration from depth-first to breadth-first—a qualitative shift in how the model can search for correct solutions.

This theoretical framing is not the paper's own language, but it is explicit in the contrast the authors draw: discrete CoT "resembles depth-first search (DFS): each sampled trace commits to a single trajectory before branching to others," while multiplex thinking enables "a more breadth-first search (BFS)-like manner" (Section 1). The BFS characterization is important because BFS is more likely to find short paths to solutions in spaces where many dead ends exist—precisely the structure of mathematical reasoning, where a single wrong algebraic manipulation can derail an otherwise correct approach.

Where Prior Approaches Fall Short

The paper identifies limitations across three distinct lines of prior work:

Discrete reasoning with RL is effective but samples poorly. The recent success of DeepSeek-R1 (Guo et al., 2025) and related work demonstrates that large-scale RL with verifiable rewards can substantially improve reasoning capabilities. However, these approaches inherit the fundamental cost structure of discrete CoT: every training rollout requires generating a full sequence of tokens, and exploring alternative reasoning paths requires separate rollouts. The paper argues this is not merely an efficiency problem but an exploration problem—discrete rollouts struggle to discover correct solutions that require navigating through multiple uncertain decision points because the probability of randomly sampling all the right decisions in a single trajectory is prohibitively low. As the authors put it, discrete methods "suffer from high computation cost of generating long sequences by conducting a depth-first style decoding to reach solutions" (Section 6).

Continuous reasoning is deterministic and incompatible with RL. A line of recent work—which the authors cite as direct motivation—proposes replacing discrete thinking tokens with continuous vectors that encode richer information. COCONUT (Hao et al., 2025) uses the transformer's last hidden states as soft tokens, training models to reason in a continuous latent space. Soft Thinking (Zhang et al., 2025) constructs concept tokens by taking the probability-weighted average of vocabulary embeddings: $c_i = \sum_{k \in V} p_i(k) e(k)$ , where $p_i$ is the model's next-token distribution and $e(k)$ is the embedding for token $k$ . This preserves the embedding space prior and avoids the representational misalignment issues that arise when using hidden states as inputs—a problem the paper notes can cause "catastrophic forgetting" in larger models (Xu et al., 2025).

However, these continuous token methods share a critical flaw: they are deterministic. Given a fixed context, the mapping from the logit distribution to the continuous token is a deterministic function (probability-weighted averaging). This collapses what was a rich distribution over the vocabulary into a single point estimate. The consequence is twofold. First, deterministic continuous tokens produce identical rollouts every time, eliminating the stochastic exploration that RL depends on. Second, they lose the probabilistic semantics that make discrete sampling interpretable and optimizable—you cannot easily compute the log-probability of a deterministic continuous trajectory, which makes it awkward to optimize with standard policy gradient methods.

Some recent work attempts to inject stochasticity into continuous reasoning. The paper cites two approaches: injecting external Gaussian noise into the logits (Butt et al., 2025) and applying the Gumbel-Softmax trick to generate stochastic soft tokens (Wu et al., 2025), the latter being the "Stochastic Soft Thinking" baseline used in the paper's experiments. But the authors argue these are fundamentally additive—they patch noise onto a deterministic framework rather than building stochasticity into the representation itself. As a result, they do not induce a well-defined probability distribution over continuous trajectories, making them difficult to optimize with likelihood-based RL objectives.

Parallel reasoning methods scale cost linearly with paths. Self-consistency (Wang et al., 2023), Best-of-N with outcome or process rewards (Cobbe et al., 2021; Lightman et al., 2023), and Tree-of-Thought (Yao et al., 2023) all improve reasoning by sampling multiple discrete trajectories and aggregating their results. These are effective but expensive: the cost grows approximately linearly with the number of paths, since each path requires generating a full sequence. The paper positions multiplex thinking as "a complementary dimension to existing parallel reasoning strategies" because it operates at the per-step token level rather than the outer-loop sampling level. In principle, one could run self-consistency over multiplex trajectories—replacing $N$ discrete rollouts with $N$ multiplex rollouts—and benefit from the improved exploration within each trajectory on top of the diversity across trajectories.

How This Paper Positions Itself

The paper's positioning is clean: it aims to fuse the information density of continuous representations with the stochastic semantics of discrete sampling—getting the best of both worlds while avoiding their respective limitations. The key insight is that continuous tokens need not be deterministic. By independently sampling K tokens from the model's own distribution and aggregating their embeddings, you obtain a continuous token that:

Preserves stochasticity: each sampling step is a random draw from the model's next-token distribution, so the multiplex token at a given position is a random variable rather than a fixed function of the input. This enables on-policy exploration and trial-and-error learning.
Induces a tractable probability distribution: because the K samples at each step are independent, the joint probability of the multiplex token factorizes as the product of the individual token probabilities (Equation 3 in Section 3.1). This means you can compute exact log-probabilities of multiplex trajectories and optimize them directly with likelihood-ratio RL objectives—something impossible with deterministic continuous tokens.
Adapts to the model's confidence: when the model is highly confident (low entropy, peaked distribution), the K independent samples will likely be identical, and the multiplex token collapses to a standard discrete token embedding. When the model is uncertain (high entropy), the K samples will be diverse, and the multiplex token encodes a richer mixture of possibilities. The representation is self-adaptive—no external mechanism decides when to be "continuous" versus "discrete."

This third property is particularly elegant and largely underexplored in the paper's analysis, but it is visible in the qualitative example (Figure 6). The trajectory alternates between consensus steps (all three samples identical, no highlighting) and exploration steps (diverse samples, highlighted in yellow/purple/red). The exploration steps consistently appear at high-entropy decision points—"these" vs. "the," "first" vs. "different," "inclusion" vs. "combin"—exactly where a human reasoner would be considering alternatives. This is not hand-crafted behavior; it emerges naturally from the sampling-and-aggregation mechanism operating on the model's own uncertainty.

The paper explicitly frames itself as bridging the gap between discrete and continuous reasoning, with the central theoretical claim that multiplex thinking provides "a well-defined probability distribution over complete reasoning trajectories, enabling direct RL optimization over multiplex rollouts without paying the full token cost of long discrete CoT" (Section 1, contribution bullet 2). This is the through-line: multiplex thinking is not just a representation trick—it is a reasoning paradigm that unifies the exploration power of stochastic sampling with the compactness of continuous representations, making both more efficient.

3. Technical Approach

This is primarily a methods paper whose core idea is that you can replace the standard discrete token-sampling step in chain-of-thought reasoning with a stochastic continuous token that aggregates multiple independent samples, preserving both the probabilistic semantics needed for RL optimization and the information density of continuous representations while naturally adapting to the model's own confidence.

3.1 Reader Orientation

What the system is: a reasoning mechanism that, at each thinking step, samples K alternative tokens from the model's output distribution, converts them to their vocabulary embeddings, and averages them into a single "multiplex token" vector that feeds back as input to the next step — instead of committing to one discrete token. What problem it solves: discrete chain-of-thought reasoning wastes the rich probability information in the model's next-token distribution by discarding all alternatives when it picks a single token, and existing continuous-token approaches lose the stochasticity that reinforcement learning needs for exploration. The solution's "shape" is a token-level branching-and-merging operation that widens exploration locally (K-way branching at each step) while keeping sequence length unchanged (one multiplex token per step), giving breadth-first dynamics at depth-first cost.

3.2 Big-Picture Architecture (Diagram in Words)

The system has four major components that operate in a tight loop:

The base language model ( $\pi_\theta$ ) — a pretrained LLM (DeepSeek-R1-Distill-Qwen-1.5B or 7B) that, at each step, takes the current context and produces a next-token probability distribution over the vocabulary. This is the same model used for both multiplex thinking and discrete baselines.
The K-way independent sampler — given the model's next-token distribution, this component draws K independent discrete tokens ( $k_{i,1}, \dots, k_{i,K}$ ) according to those probabilities. Each draw is i.i.d. from the same distribution, with no dependence between samples.
The multiplex token constructor — takes the K sampled token IDs, maps each to its vocabulary embedding via the model's embedding matrix $E$ , and aggregates them into a single continuous vector $c_i$ . By default, this aggregation uses a reweighting scheme based on the model's LM-head probabilities, but a simpler unweighted average works nearly as well.
The GRPO reinforcement learning loop — during training, the model generates complete multiplex thinking traces (sequence of multiplex tokens followed by discrete answer tokens), receives a reward based on answer correctness, and updates its parameters to maximize expected reward using a likelihood-ratio objective. The loop uses Group Relative Policy Optimization (GRPO) with 8 rollouts per question, temperature 1.0, top-p 1.0, and no KL or entropy penalties.

Information flows as follows: a prompt enters the model → the model outputs a next-token distribution → the sampler draws K independent tokens → the multiplex constructor aggregates their embeddings → the resulting multiplex token becomes part of the input context for the next step → this repeats until the highest-probability sampled token is the special end-of-thinking token [eot] → the model then generates discrete answer tokens normally → a verifiable reward function checks the final answer → GRPO updates the model parameters.

3.3 Roadmap for the Deep Dive

First, the multiplex token construction (Section 3.1): the mathematical definition of how K discrete samples become one continuous token, the two aggregation strategies (uniform averaging vs. LM-head reweighting), and how the self-adaptive behavior emerges from the sampling process.
Second, the probability factorization: why independent sampling induces a tractable log-probability for the entire multiplex trajectory, and why this matters for RL optimization.
Third, the reinforcement learning objective (Section 3.2): how the factored log-probability plugs into the GRPO update, the reward structure, and the training hyperparameters.
Fourth, the entropy analysis (Section 3.3): a formal comparison of the exploration capacity of multiplex tokens versus discrete tokens, showing the linear-in-K expansion of the effective action space.
Fifth, the inference-time and training design choices: stopping criteria, the transition from thinking to answering, and why rule-based heuristics were deliberately avoided.

3.4 Detailed, Sentence-Based Technical Breakdown

This is a methods paper that proposes multiplex thinking as a token-level mechanism for stochastic continuous reasoning. The core idea is to replace the standard "sample one discrete token" operation with "sample K discrete tokens, embed them, and aggregate the embeddings," creating a continuous representation that retains both the stochasticity of discrete sampling (for exploration) and the information density of continuous vectors (for efficiency). The mechanism is self-contained — it does not require changes to the model architecture, only to how tokens are fed back as input during the thinking phase.

Multiplex Token Construction

At reasoning step $i$ , the language model $\pi_\theta$ has processed the question tokens $e(q)$ and all previous multiplex thinking tokens $c_{<i} = (c_1, \dots, c_{i-1})$ . It produces a next-token probability distribution over the vocabulary $V$ :

$p_i = \pi_\theta(\cdot \mid e(q), c_{<i})$

where $p_i \in \mathbb{R}^{|V|}$ is a probability vector (non-negative entries summing to 1), $e(q)$ is the embedded question, and $c_{<i}$ represents the sequence of multiplex tokens generated so far.

What it computes: the model's distribution over which token should come next, given the question and the reasoning trace up to this point. This is identical to what a standard discrete LLM does at every decoding step — the difference is only in what happens next.

From this distribution, we independently sample $K$ discrete tokens:

$k_{i,1}, k_{i,2}, \dots, k_{i,K} \overset{\text{i.i.d.}}{\sim} p_i$

where each $k_{i,j} \in \{1, \dots, |V|\}$ is a token index sampled according to the probabilities in $p_i$ , and the samples are independent — $k_{i,2}$ is drawn without knowing what $k_{i,1}$ was.

What it computes: K parallel "guesses" at what the next token should be, each following the model's own uncertainty. When the model is confident (e.g., $p_i(\text{"the"}) \approx 0.95$ ), all K samples will almost certainly be "the." When the model is uncertain (e.g., $p_i(\text{"first"}) \approx 0.4$ , $p_i(\text{"different"}) \approx 0.35$ , $p_i(\text{"car"}) \approx 0.25$ ), the K samples will be a diverse set reflecting that uncertainty.

These K samples are aggregated by first representing each as a one-hot vector and averaging them:

$s_i = \frac{1}{K} \sum_{j=1}^K z_{i,j}$

where $z_{i,j} \in \{0,1\}^{|V|}$ is the one-hot encoding of token $k_{i,j}$ (a vector with a 1 at index $k_{i,j}$ and zeros elsewhere), and $s_i \in [0,1]^{|V|}$ is the empirical distribution of the K samples — $s_i[v]$ is the fraction of the K samples that were token $v$ .

What it computes: a sparse vector over the vocabulary that represents which tokens were sampled and how often. If all K samples were the same token, $s_i$ is exactly a one-hot vector ( $K=1$ recovers standard discrete decoding). If the samples were diverse, $s_i$ has multiple non-zero entries, each equal to the fraction of samples that landed on that token (e.g., $2/3, 1/3$ for $K=3$ with one duplicate). In the limit $K \to \infty$ , $s_i$ converges to the true distribution $p_i$ .

The continuous multiplex token is obtained by mapping this empirical distribution through the embedding matrix $E$ with an optional vocabulary-space reweighting:

$c_i = E^\top (s_i \odot w_i)$

where $E \in \mathbb{R}^{|V| \times d}$ is the vocabulary embedding matrix (row $v$ is the $d$ -dimensional embedding of token $v$ ), $E^\top \in \mathbb{R}^{d \times |V|}$ is its transpose (mapping from vocabulary space to embedding space), $w_i \in \mathbb{R}^{|V|}$ is a per-token weight vector, $\odot$ denotes element-wise multiplication, and $c_i \in \mathbb{R}^d$ is the resulting multiplex token embedding.

What it computes: the weighted average of the embeddings of the K sampled tokens, producing a single $d$ -dimensional vector that lives in the same embedding space as standard discrete token embeddings. This vector can be fed directly as input to the next transformer layer — the model's attention mechanism and feedforward layers process it just like any other embedded token.

Why this form: using the embedding matrix as the mapping from vocabulary space to continuous space preserves the vocabulary embedding prior, meaning the multiplex token stays in the same representational space the model was pretrained on. This avoids the representational misalignment that occurs when using hidden states as continuous tokens (as in COCONUT), where the input embeddings and the prediction head operate in different spaces, potentially causing the model to "forget" its pretrained knowledge. The alternative — using the model's last hidden state directly — would require training the model to interpret its own hidden states as input, a form of representational mismatch that the paper notes can cause catastrophic forgetting in larger models.

The paper considers two choices for the weight vector $w_i$ :

Uniform averaging: $w_i[v] = 1$ for all vocabulary indices $v$ . This simplifies to:

$c_i = E^\top s_i = \frac{1}{K} \sum_{j=1}^K e(k_{i,j})$

which is simply the arithmetic mean of the embeddings of the K sampled tokens.

LM-head reweighting: $w_i[v] = K \cdot \frac{\mathbb{1}[s_i[v] > 0] \cdot \pi_\theta(v \mid e(q), c_{<i})}{\sum_{u=1}^V \mathbb{1}[s_i[u] > 0] \cdot \pi_\theta(u \mid e(q), c_{<i})}$

where $\mathbb{1}[s_i[v] > 0]$ is 1 if token $v$ appeared at least once in the K samples and 0 otherwise, and $\pi_\theta(v \mid \dots)$ is the model's assigned probability to token $v$ . This reweighting does two things: (1) it only considers tokens that were actually sampled (the indicator function), and (2) it scales each sampled token's contribution by the model's own confidence in that token, normalized over the sampled set. The factor $K$ ensures the scale matches the uniform averaging case.

What this reweighting achieves: it biases the multiplex token toward the tokens the model is more confident about, even among the sampled set. If two tokens were both sampled but the model assigns probability 0.7 to one and 0.2 to the other, the reweighted version gives the higher-probability token roughly $0.7/(0.7+0.2) \approx 0.78$ of the weight rather than $0.5$ . The paper reports that both strategies yield "comparable performance" (Section 5.5, Table 5), suggesting the core benefit comes from having multiple tokens represented rather than from the precise mixing coefficients. In the main experiments, reweighting is used by default "as it more directly reflects the model's confidence over the sampled candidates."

Self-Adaptive Behavior: When Multiplex Collapses to Discrete

A crucial property of this construction is that it is self-adaptive to the model's confidence, requiring no external mechanism to decide whether to behave discretely or continuously. This emerges from the sampling step:

When the next-token distribution $p_i$ is highly peaked (low entropy, e.g., $p_i(\text{token}_v) \approx 0.98$ ), the probability that all K independent samples land on the same token is approximately $0.98^K$ , which is close to 1 even for moderate K. The resulting $s_i$ is effectively one-hot, and $c_i \approx e(\text{token}_v)$ — indistinguishable from standard discrete token embedding.
When the next-token distribution has high entropy (e.g., multiple tokens with probabilities around 0.2–0.4), the probability of diverse samples is high. The resulting $c_i$ is a mixture of multiple token embeddings, encoding a "superposition" of alternative reasoning paths in a single vector — a continuous token that cannot be represented as any single discrete token embedding.

This means multiplex thinking naturally transitions between discrete-like and continuous-like behavior on a per-step basis, driven entirely by the model's own uncertainty. No training mechanism, threshold, or heuristic decides this — it is a direct consequence of the independent sampling plus aggregation procedure. The qualitative visualization in Figure 6 makes this visible: steps where all three sampled tokens agree appear as plain text (consensus), while steps with diverse samples are highlighted, and these exploration steps cluster at the semantically meaningful decision points in the reasoning (e.g., choosing between "first," "different," and "car" as the next token when describing an inclusion-exclusion approach).

Probability Factorization over Multiplex Trajectories

Because the K samples at each step are drawn independently from the same distribution $p_i$ , the joint probability of a specific multiplex token — meaning the specific set of K sampled tokens $\{k_{i,1}, \dots, k_{i,K}\}$ — factorizes cleanly. The log-probability of the entire multiplex thinking trace $c = (c_1, \dots, c_L)$ is:

$\log \pi(c \mid e(q)) = \sum_{i=1}^{|c|} \sum_{j=1}^K \log \pi_\theta(k_{i,j} \mid e(q), c_{<i})$

where $|c|$ is the number of multiplex thinking steps (the length of the thinking trace), $K$ is the multiplex width, $k_{i,j}$ is the $j$ -th token sampled at step $i$ , and $\pi_\theta(k_{i,j} \mid e(q), c_{<i})$ is the probability the model assigns to that token given the question and previous multiplex tokens.

What it computes: the log-likelihood of a multiplex trajectory is simply the sum of the log-likelihoods of all $K \times |c|$ discrete tokens that were sampled to construct it. There is no approximation, no variational bound, no Monte Carlo estimate — it is exact because of the independence assumption.

Why this form matters for RL: standard policy gradient methods (including GRPO) require computing the log-probability of the actions taken during a rollout. For discrete CoT, this is straightforward — each action is a token, and its log-probability is a scalar from the model's output distribution. For deterministic continuous tokens (like the original Soft Thinking), this is impossible — the continuous token is a deterministic function of the logits, and there is no notion of "probability of this continuous token" that can be differentiated through. Multiplex thinking solves this by preserving the sampling step: the continuous token is constructed from discrete samples, each of which has a well-defined log-probability under the model. This is the key bridge that enables on-policy RL optimization over continuous reasoning traces.

Reinforcement Learning Objective

The paper optimizes the multiplex thinking model using Group Relative Policy Optimization (GRPO; Shao et al., 2024), a variant of policy gradient methods adapted for language model fine-tuning. The objective is to maximize the expected reward of the final answer $y$ generated after the multiplex thinking trace $c$ :

$\mathcal{J}_{\text{RL}}(\theta) = \mathbb{E}_{\substack{(q, y^*) \sim \mathcal{D}, \\ c \sim \pi_\theta(\cdot \mid q), \\ y \sim \pi_\theta(\cdot \mid q, c)}} \left[ (\log \pi_\theta(c \mid e(q)) + \log \pi_\theta(y \mid e(q), c)) \cdot v(y, y^*) \right]$

where $\mathcal{D}$ is the training dataset of question-answer pairs $(q, y^*)$ , $c \sim \pi_\theta(\cdot \mid q)$ denotes sampling a multiplex thinking trace from the model (using the mechanism described above), $y \sim \pi_\theta(\cdot \mid q, c)$ denotes sampling the final answer tokens given the question and the thinking trace, $\log \pi_\theta(c \mid e(q))$ is the log-probability of the thinking trace (from the factorization above), $\log \pi_\theta(y \mid e(q), c)$ is the log-probability of the answer tokens, and $v(y, y^*) \in \{0, 1\}$ is the verifiable reward function that returns 1 if the extracted answer $y$ matches the ground-truth $y^*$ and 0 otherwise.

What it computes: this is a standard likelihood-ratio (REINFORCE-style) objective. The model generates a complete trajectory — thinking trace followed by answer — and receives a binary reward based on answer correctness. The log-probability of the trajectory is scaled by the reward: trajectories that produce correct answers get their log-probability multiplied by +1 (reinforcing them), while incorrect trajectories get multiplied by 0 (effectively ignored, since the reward is binary and zero). The expectation is over the randomness in both the thinking trace and the answer sampling.

Why this form works: the key enabler is that $\log \pi_\theta(c \mid e(q))$ is computable and differentiable. In standard discrete RL for CoT, this term would be the sum of log-probabilities of each discrete thinking token. In multiplex thinking, it is the sum of log-probabilities of all K sampled tokens at each step. The only difference is the factor of K — each thinking step contributes K terms instead of 1 — but the mathematical structure is identical. The GRPO algorithm handles the optimization details (clipping, advantage normalization, group-relative comparisons) exactly as it would for discrete tokens.

Training hyperparameters (Table 6, Appendix A.1.1):

Training dataset: DeepScaleR-Preview-Dataset (~40,000 question-answer pairs)
Training steps: 300
Global batch size: 128 questions
PPO mini batch size: 128
Rollout number: 8 (each question gets 8 independent multiplex rollouts per training step)
Learning rate: $1 \times 10^{-6}$
Optimizer: AdamW (implicit from the GRPO framework)
Sampling temperature: 1.0 (no temperature scaling during training rollouts)
Sampling top-p: 1.0 (no nucleus filtering during training rollouts)
Multiplex width $K$ : 3 (default for main experiments)
Maximum prompt length: 1024 tokens
Maximum response length: 4096 tokens (covers both thinking and answer phases)
KL loss coefficient: 0 (no KL penalty toward the reference model)
Entropy loss coefficient: 0 (no bonus for maintaining high entropy)
Model data type: bfloat16

Design choice — zero KL and entropy penalties: the paper explicitly sets both to zero, meaning the model is trained purely on reward maximization without regularization toward a reference policy or toward high entropy. This is a deliberate choice that trusts the multiplex representation to maintain sufficient exploration on its own (an assumption validated by the entropy analysis in Section 5.4, which shows multiplex training has less entropy collapse than discrete RL). Adding a KL penalty would anchor the model to its pretrained behavior; removing it allows the model to freely shift probability mass toward correct reasoning patterns. The entropy penalty is similarly omitted because the multiplex mechanism already provides structured exploration through its K-way sampling.

Stopping Criteria: Transitioning from Thinking to Answering

During inference (and during training rollouts), the model must decide when to stop multiplex thinking and begin generating the final answer. The paper uses a simple mechanism: the transition is triggered when the discrete token with the highest probability under the model's next-token distribution is the special end-of-thinking token, denoted [eot].

At each thinking step $i$ , the model produces the distribution $p_i = \pi_\theta(\cdot \mid e(q), c_{<i})$ before any sampling occurs. The system checks whether $\arg\max_v p_i(v)$ equals the token ID for [eot]. If so, the thinking phase ends, and subsequent tokens are generated using standard discrete decoding (normal autoregressive sampling, one token at a time, no aggregation). The K-way sampling and aggregation only applies during the thinking phase; the answer phase is standard discrete generation.

Why this criterion rather than heuristics: the paper explicitly notes (Appendix A.1, "Stopping Criteria") that they considered training-free heuristics, such as monitoring for consecutive low-entropy tokens as an early-stopping signal (used in some other continuous reasoning works), but found these heuristics "introduce artificial patterns that the model tends to exploit during RL optimization, resulting in training instability and the generation of incoherent content." The model learns to game any fixed stopping rule to maximize reward without actually reasoning. By using the model's own highest-probability token as the trigger, the RL objective naturally regulates the thinking process: if the model thinks indefinitely without producing the [eot] token and a final answer, it never receives a reward, so it learns to produce [eot] at appropriate points. This is a self-regulating mechanism — no hand-crafted budget or threshold is needed.

A subtle consequence of this design: the [eot] token is not sampled during the thinking phase — it is used purely as a control signal based on the argmax of the probability distribution. The K-way sampling at each step draws from the full distribution, but the stop/continue decision is deterministic (argmax), not stochastic. This prevents the model from accidentally terminating early due to a low-probability sample of [eot] when the model is actually uncertain, and it ensures the thinking phase continues as long as the model believes (in terms of its peak probability) that more reasoning is needed.

Entropy Analysis: Why Multiplex Tokens Expand Exploration

To formalize the exploration advantage of multiplex thinking, the paper compares the entropy of a multiplex token to that of a standard discrete token. For standard CoT, the discrete token $t_i$ is sampled from the distribution $p_i$ , and the entropy of this single-step sampling is the Shannon entropy:

$H_{\text{CoT}}(i) = - \sum_{v \in V} \pi_\theta(v \mid q, t_{<i}) \log \pi_\theta(v \mid q, t_{<i})$

where the sum is over all vocabulary items $v \in V$ , and $\pi_\theta(v \mid \dots)$ is the probability assigned to each token. This is the standard measure of uncertainty at a single decoding step — higher entropy means the model is more uncertain about which token to pick next.

What it computes: the expected information content (in nats or bits, depending on the log base) of sampling a single token from the distribution. For a peaked distribution (e.g., one token with probability 0.99), entropy is near zero — there is almost no uncertainty. For a uniform distribution over $|V|$ tokens, entropy is maximal at $\log |V|$ .

For multiplex thinking, the authors conceptualize the generation of a multiplex token as a single integrated action that selects a composite outcome $(k_{i,1}, \dots, k_{i,K})$ — an ordered K-tuple of token indices — from the augmented action space $|V|^K$ . The joint entropy of this composite outcome is:

$H(\mathcal{K}_i) = K \cdot H(\pi_\theta(q, c_{<i}))$

where $\mathcal{K}_i = \{k_{i,1}, \dots, k_{i,K}\}$ is the set of K sampled tokens, $H(\pi_\theta(q, c_{<i}))$ is the Shannon entropy of the model's next-token distribution at step $i$ (identical to $H_{\text{CoT}}(i)$ but computed over the multiplex context), and the multiplication by $K$ comes from the independence of the K samples — the entropy of K independent draws is simply K times the entropy of a single draw.

What it computes: the total uncertainty in the multiplex token, treated as a single composite random variable. If the model's per-token entropy at step $i$ is 2.0 nats, then a multiplex token with $K=3$ has a joint entropy of 6.0 nats, representing a substantially larger "exploration budget" at that step.

Why this matters for RL: the effective action space size scales exponentially in K. A discrete CoT model chooses from $|V|$ possible actions at each step (e.g., ~32,000 tokens). A multiplex thinking model chooses from $|V|^K$ possible composite actions at each step (e.g., with $K=3$ and $|V|=32,000$ , approximately $3.3 \times 10^{13}$ possible outcomes). This does not mean the model explicitly enumerates all possibilities — it only samples K of them — but the entropy metric captures the fact that the model is "considering" (in terms of probability mass) a much larger space of next-step continuations simultaneously. This expanded exploration is what the paper credits for the improved Pass@k scaling (Figure 2), where multiplex thinking maintains a widening gap over discrete baselines as k grows, particularly on hard problems.

The RL Training Loop: End-to-End Process

Putting the pieces together, the training loop for multiplex thinking proceeds as follows at each training step:

Rollout generation: for each question in the batch (global batch size 128), the model generates 8 independent multiplex thinking rollouts (total 1024 rollouts per step). Each rollout:
- Processes the prompt tokens (up to 1024 tokens) through the transformer to get the first next-token distribution.
- Enters the multiplex thinking loop: at each step, samples $K=3$ tokens, constructs the multiplex token via $c_i = E^\top(s_i \odot w_i)$ with LM-head reweighting, feeds $c_i$ back as input.
- Checks whether $\arg\max$ of the current distribution is [eot]; if so, exits the thinking loop.
- Generates answer tokens using standard discrete autoregressive sampling until the end-of-sequence token.
- The total sequence (thinking + answer) is capped at 4096 tokens.
Reward computation: for each rollout, the final answer is extracted and compared to the ground-truth answer $y^*$ using a verifiable reward function $v(y, y^*) \in \{0, 1\}$ (binary reward: 1 for correct, 0 for incorrect).
Advantage computation (GRPO-specific): within each group of 8 rollouts for the same question, the rewards are used to compute advantages — rollouts with above-average performance in their group receive positive advantages, below-average receive negative advantages. This group-relative normalization reduces variance compared to using absolute rewards.
Policy gradient update: the model parameters $\theta$ are updated to maximize the GRPO objective, which involves computing the gradient of the log-probability of each rollout's trajectory (thinking trace + answer) weighted by the advantage, with clipping to prevent overly large updates. The log-probability of the thinking trace is computed using the factorization $\sum_i \sum_j \log \pi_\theta(k_{i,j} \mid \dots)$ — the sum of log-probabilities of all $K \times L$ discrete tokens sampled during the thinking phase.
Repeat: steps 1–4 are repeated for 300 training steps. Validation is performed every 25 steps using Pass@4 on MATH-500.

Design Choices and Their Justifications

Why sampling-based rather than deterministic continuous tokens: the paper argues that deterministic continuous tokens (probability-weighted embedding mixtures in Soft Thinking, or hidden-state tokens in COCONUT) collapse the token-level policy distribution into a single point. This eliminates stochastic exploration, which is essential for RL to discover new reasoning strategies through trial and error. Multiplex thinking preserves the sampling step, making each rollout a stochastic draw from the model's distribution — identical in spirit to discrete sampling, but operating in a continuous embedding space.

Why independent sampling rather than sequential: the K samples at each step are drawn independently rather than sequentially conditioned on previous samples. This independence is what makes the log-probability factorize cleanly (product of marginals, no conditional terms) and enables the simple entropy scaling $K \cdot H$ . If samples were drawn sequentially (sampling $k_1$ , then feeding it back to get a distribution for $k_2$ , etc.), the probability would be $\pi(k_1) \cdot \pi(k_2 \mid k_1) \cdots$ , and the joint entropy would be more complex (chain rule of entropy, not simple multiplication). The independence assumption trades some fidelity to the true joint distribution for tractability — an acceptable simplification since the goal is exploration, not exact inference over K-tuples.

Why K is small (2–6): the paper sweeps $K \in \{1, 2, 3, 6\}$ and finds that the largest jump in performance comes from moving from $K=1$ (discrete) to $K=2$ , with diminishing returns thereafter (Section 5.2, Figure 3). This is consistent with the intuition that the most valuable exploration comes from considering the top few alternatives at each step — tokens with negligible probability are unlikely to be correct and adding them to the mixture primarily adds noise. The default $K=3$ is chosen as a practical balance between exploration benefit and computational cost (sampling K tokens and averaging K embeddings adds negligible overhead since all K samples come from the same logits distribution).

Why no KL or entropy penalty: removing these regularization terms is a vote of confidence in the multiplex mechanism's built-in exploration. The hypothesis (supported by the entropy analysis in Section 5.4) is that the K-way sampling provides sufficient diversity to prevent premature policy collapse, making explicit entropy bonuses unnecessary. The zero KL penalty similarly trusts that the multiplex representation's grounding in vocabulary embeddings prevents the model from drifting into degenerate continuous spaces.

Why LM-head reweighting by default: while both uniform averaging and LM-head reweighting produce comparable results (Table 5), reweighting has the conceptual advantage that it respects the model's own confidence ordering among sampled tokens. If the model assigns probability 0.6 to token A and 0.3 to token B, and both happen to be sampled, reweighting ensures A dominates the mixture — consistent with the model's belief that A is the more likely correct continuation. Uniform averaging would give them equal weight despite the model's preference. The empirical similarity of the two suggests the model is robust to the precise mixing coefficients, likely because the transformer's attention mechanism can learn to extract the relevant signal regardless.

4. Key Insights and Innovations

Innovation 1: Reframing the Discrete vs. Continuous Reasoning Tradeoff as a Sampling Problem

The paper's most distinctive conceptual move is redefining the bottleneck in continuous reasoning not as a representation problem, but as a sampling problem. Prior continuous token approaches — COCONUT (Hao et al., 2025) using hidden states and Soft Thinking (Zhang et al., 2025) using probability-weighted embedding mixtures — framed the challenge primarily as one of information preservation: how to encode the rich next-token distribution into a compact representation without losing the expressive power the model would have had by considering all alternatives. This framing led naturally to deterministic solutions: take the expectation over the vocabulary, or use the model's internal state, both of which produce a fixed output for a given input. The field had implicitly accepted that continuous reasoning meant deterministic reasoning, since "continuity" and "stochasticity" seemed in tension — how do you define a probability distribution over a continuous vector in a way that's both meaningful and tractable?

Multiplex Thinking rejects this framing entirely. The paper's insight is that you don't need to define a probability distribution over the continuous token itself — you define it over the discrete tokens that generated it, and the continuous token inherits its stochasticity from them. This is a subtle but profound shift. In Soft Thinking, the attention is on the output (the continuous token), which is deterministic, so there is no natural notion of "log-probability of this trajectory" that can be plugged into a policy gradient. In Multiplex Thinking, the attention is on the process (the K independent discrete samples), each of which has an unambiguous log-probability under the model, so the log-probability of the trajectory is simply the sum of all those token-level log-probabilities. The continuous token is a derived object — it carries the representational benefits, but the stochastic semantics live in the discrete layer beneath it.

This diagnostic reframing matters far beyond the specific method. It reveals that the real obstacle to RL-compatible continuous reasoning was never about representation quality — it was about maintaining the link between continuous representations and the discrete probability distributions that RL objectives need. Prior work tried to add stochasticity as an afterthought (Gumbel noise in Stochastic Soft Thinking; Butt et al., 2025; Wu et al., 2025), effectively bolting randomness onto a deterministic base. Multiplex Thinking builds stochasticity into the foundation by preserving the sampling operation at each step. The significance is not the specific aggregation formula (averaging embeddings) but the architectural principle: continuous reasoning tokens should be stochastic functionals of discrete samples, not deterministic functions of logit distributions. This is a fundamental conceptual shift, not an incremental improvement — it reopens the continuous reasoning literature to the entire RL optimization toolbox that previous methods had implicitly walled off.

Evidence for this reframing's importance is visible in the comparison between Multiplex Thinking and Discrete RL (Table 1, Figure 2): both use the identical GRPO training setup, data, and hyperparameters, but Multiplex Thinking consistently outperforms because its exploration space at each step is fundamentally richer, not because it has access to better reward signals or optimization techniques. The RL objective itself is unremarkable — it's standard GRPO with zero KL and entropy penalties. The gains come entirely from the representation's ability to maintain diverse exploration during training, which the entropy analysis (Table 4) quantifies: multiplex training shows 5.82–7.09% entropy reduction versus 9.44% for discrete RL, meaning the policy retains more of its initial exploration variance throughout training.

Innovation 2: Self-Adaptive Reasoning Without Heuristics — Confidence Gating Emerges from the Sampling Mechanism

A second distinctive contribution is the demonstration that a single, unified mechanism can naturally alternate between discrete and continuous behavior on a per-step basis, driven purely by the model's own uncertainty, without any explicit gating, threshold, or learned controller. This is not a claim the paper makes explicitly as a theoretical contribution — it emerges from the design rather than being the headline — but it is arguably the most elegant property of Multiplex Thinking and one that distinguishes it from essentially all prior work on adaptive reasoning.

The field has long recognized that not all reasoning steps are equally hard, and that resources should be allocated adaptively. Prior approaches to adaptive reasoning fall into two categories, both requiring explicit decision mechanisms. Learned controllers: methods like ThreadWeaver (Lian et al., 2025) or adaptive parallel reasoning (Pan et al., 2025) train separate modules or policies to decide when to branch, when to aggregate, and how many paths to explore. These add complexity, separate training objectives, and potential failure modes. Heuristic controls: many continuous reasoning works use fixed rules — stop reasoning after N consecutive low-entropy tokens, branch when confidence falls below a threshold, etc. The paper explicitly notes (Appendix A.1) that such heuristics "introduce artificial patterns that the model tends to exploit during RL optimization, resulting in training instability and the generation of incoherent content" — the model learns to game the heuristic rather than reason better.

Multiplex Thinking achieves adaptive behavior with zero additional mechanism. The sampling-and-aggregation operation has two natural regimes that emerge from a single mathematical procedure: when the distribution is peaked, independent samples all land on the same token, and $c_i$ is a one-hot embedding (discrete behavior); when the distribution is flat, samples differ, and $c_i$ is a continuous mixture (exploration behavior). There is no threshold, no learned gating network, no hand-crafted rule — just the probabilistic fact that $K$ independent draws from the same distribution are more likely to agree when that distribution is concentrated.

The qualitative visualization in Figure 6 makes this visible in a way that is intellectually compelling: exploration steps (highlighted) cluster at the semantically meaningful decision points — "first" vs. "different" vs. "car" when the model is choosing how to describe its approach. Consensus steps (unhighlighted, all three samples identical) appear during more routine transitions. This is not behavior the model was trained to produce; it is a direct mathematical consequence of the sampling mechanism operating on the model's own uncertainty. The model's epistemic state at each step automatically determines whether the next token will behave like a discrete choice or a continuous superposition.

This is a fundamental contribution to the design principles for adaptive reasoning — not because it achieves a new state-of-the-art (though it does), but because it demonstrates that adaptivity can be emergent from a single well-designed primitive rather than engineered through separate control modules. This is the kind of insight that changes how future systems are designed: rather than building explicit "branch/when-uncertain" controllers, one might design reasoning primitives whose natural operating characteristics automatically allocate exploration where it is needed, driven by the model's own uncertainty signal.

Evidence for the importance of this adaptivity is visible in the comparison between Multiplex Thinking-I (inference-only, no RL training) and Stochastic Soft Thinking on the 7B scale (Table 2). Both are training-free methods that operate on the same base model, but Multiplex Thinking-I matches or exceeds Stochastic Soft Thinking on most benchmarks. The key difference is precisely this self-adaptive behavior: Stochastic Soft Thinking applies a uniform stochasticity injection (Gumbel-Softmax with fixed temperature) regardless of the model's confidence, while Multiplex Thinking's stochasticity is modulated by the model's own distribution — sharp when confident, diverse when uncertain.

Innovation 3: The Inference-Only Competitiveness Reveals That Multiplex Representations Are Intrinsically Beneficial, Not Just an RL Amplifier

A diagnostic finding that the paper presents almost in passing — but which carries substantial conceptual weight — is that Multiplex Thinking without any training outperforms Discrete CoT and is competitive with Stochastic Soft Thinking, the previous best training-free continuous method (Table 2, "Multiplex Thinking-I"). On the 7B backbone, Multiplex Thinking-I scores 20.5 vs. 20.3 on AIME 2024, 19.6 vs. 19.1 on AIME 2025, and 48.6 vs. 47.9 on AMC 2023, all relative to Stochastic Soft Thinking. These margins are small, but that is precisely the point: a method designed primarily for RL compatibility matches or exceeds a method designed specifically for inference-time use, before any optimization is applied.

This matters because it decouples two claims that could have been confounded. One possible narrative is: "Multiplex Thinking works because it enables better RL exploration, which in turn produces a better-trained model." Under this narrative, the gains are entirely due to the RL optimization — the multiplex representation is just a vehicle for better gradient signals. The inference-only results falsify this narrow interpretation. Multiplex representations help even when no learning occurs, suggesting that the information density of aggregating multiple sampled tokens into a single continuous vector is intrinsically beneficial for the model's reasoning process, independent of any optimization effects.

This connects to a deeper question that the paper does not fully address but that its results raise: why should a continuous mixture of token embeddings be better than the discrete tokens themselves? The model was pretrained on discrete tokens; its representations and attention patterns were optimized for sequences of one-hot embeddings. A multiplex token is a vector that almost certainly does not correspond to any single vocabulary item — it lives in the convex hull of vocabulary embeddings but typically not at any vertex. The finding that this off-manifold representation helps (even without training) suggests that the model's pretrained transformer layers are capable of processing "superposed" tokens in meaningful ways, extracting information about the alternative paths they encode. This is a significant empirical finding about the emergent capabilities of pretrained transformers — they can reason over continuous mixtures of token embeddings in ways that preserve and leverage the uncertainty information, even though they were never trained to do so.

The practical implication is substantial: Multiplex Thinking can be deployed as a pure inference-time method to improve reasoning on any compatible model without any fine-tuning, serving as a drop-in replacement for standard CoT decoding with improved accuracy and shorter sequences. The fact that it also enables subsequent RL fine-tuning is a bonus, not the whole story.

Evidence for this intrinsic benefit is clearest in Table 2, but also in the sequence length analysis (Figure 5, Table 3): Multiplex Thinking-I-4k (4,096 token budget, inference-only) matches the accuracy of Discrete CoT-5k (5,120 token budget), a 20% sequence length reduction with no accuracy loss. This efficiency gain is present even without any training, confirming that it arises from the representation itself — each multiplex token carries more information than a discrete token, so fewer steps are needed to achieve the same reasoning outcome.

Innovation 4: Characterizing Multiplex Width as a Controllable Compute-Exploration Knob with Diminishing Returns

The paper provides the first systematic study of how the width of continuous token construction (the number K of discrete samples aggregated per step) trades off against downstream performance, establishing that (a) the transition from K=1 (discrete) to K≥2 (multiplex) is the critical regime change, and (b) returns diminish rapidly beyond K=2–3 (Section 5.2, Figure 3). While this might seem like a routine hyperparameter ablation, it carries conceptual weight because it characterizes multiplex width as a fundamental scaling dimension analogous to, but distinct from, sequence length — a new axis along which reasoning compute can be allocated.

Prior work on continuous reasoning tokens treated the construction as a fixed operation: Soft Thinking uses the full vocabulary-weighted expectation (effectively K → ∞), COCONUT uses a single hidden state, Stochastic Soft Thinking uses Gumbel-Softmax with a single draw. None of these prior methods studied the effect of varying the number of samples aggregated, because their formulations either had no such parameter (deterministic methods) or fixed it by construction (single-sample stochastic methods). Multiplex Thinking introduces K as a freely tunable hyperparameter that controls the exploration budget per reasoning step, creating a new dimension for compute-optimal allocation.

The finding that the K=1 to K=2 jump is by far the largest — e.g., on AMC 2023 (7B), from 44.7% (K=1) to 49.6% (K=2), a +4.9 percentage point gain, while K=3 and K=6 add only modest further improvements — has a clear interpretation: the primary benefit of multiplex thinking comes from breaking the "single-token bottleneck," not from exploring many alternatives per step. A single discrete token commits irrevocably; adding even one alternative creates the possibility of encoding uncertainty. Adding a second, third, or sixth alternative provides rapidly diminishing marginal value because the most informative alternatives (those with the highest probability under the model's distribution) are captured by the first few samples.

This has practical implications for scaling test-time compute. The paper's Pass@k analysis (Figure 2) shows that sampling budgets are better spent on more multiplex trajectories (larger k, more parallel rollouts) rather than wider individual tokens (larger K), since the per-token width saturates quickly while trajectory-level diversity continues to provide gains. This is a form of allocation insight: within a fixed inference budget, spending compute on sampling more complete rollouts (with moderate K) yields more benefit than spending it on wider per-step exploration within a single rollout.

The entropy reduction analysis (Table 4, Section 5.4) provides a mechanistic explanation for this pattern: higher K prevents entropy collapse during RL training (entropy reduction of 5.82–7.09% for K≥2 vs. 9.44% for K=1), but the exploration benefit plateaus because the policy's entropy is bounded below by the problem difficulty — once enough exploration is maintained to avoid premature commitment, additional width adds diversity within steps that the model cannot productively leverage. This connects the width-ablation result to the difficulty-dependent scaling observed in the Pass@k curves (Figure 2): on easy problems (MATH-500), performance saturates quickly regardless of method because there are few truly ambiguous reasoning steps where multiplex exploration can help; on hard problems (AIME 2025), the gap between multiplex and discrete widens with k because the longer, more complex reasoning traces contain more decision points where the single-token bottleneck would be most damaging.

Evidence: Figure 3 and Table 4, interpreted jointly with Figure 2's Pass@k curves and Figure 6's qualitative visualization showing how exploration steps cluster at high-entropy decision points.

5. Experimental Analysis

Evaluation Methodology

Dataset. The training set is DeepScaleR-Preview-Dataset (Luo et al., 2025), consisting of approximately 40,000 unique problem-answer pairs. Evaluation uses six challenging math reasoning benchmarks: AIME 2024 (Veeraboina), AIME 2025 (Zhang & Math-AI, 2025), AMC 2023, MATH-500 (Hendrycks et al., 2021), Minerva Math (Lewkowycz et al., 2022), and OlympiadBench (He et al., 2024). These span a range of difficulties from standard competition math (MATH-500) to elite olympiad problems (AIME, OlympiadBench), allowing the paper to assess both saturated and unsaturated performance regimes.
Base model(s). All experiments use DeepSeek-R1-Distill-Qwen at two scales: 1.5B and 7B parameters. These are open-source reasoning backbones that have already been distilled from a larger R1-style model, giving them non-trivial baseline reasoning capabilities (e.g., 15.7% on AIME 2024 for the 7B model under discrete CoT) while leaving substantial room for improvement. The two-scale design allows testing whether multiplex thinking's benefits depend on model capacity — a question the results answer affirmatively: gains are larger and more consistent at 7B than at 1.5B.
Metrics. The primary metric throughout is Pass@1 accuracy (%) — the fraction of problems for which the model's single best answer is correct, averaged over 64 independent runs. For test-time scaling analysis, Pass@k is computed for k ∈ {1, 2, 4, ..., 1024}, measuring the probability that at least one correct solution exists among k sampled trajectories (Chen et al., 2021). Pass@k values are estimated by bootstrapping 1,000 times over a total of 1,024 runs per problem. Sequence length (number of generated tokens) is also tracked as a secondary efficiency metric.
Baselines. Three distinct categories of methods serve as comparisons:
- Discrete CoT: the backbone models using standard discrete chain-of-thought decoding without any additional training.
- Discrete RL: the backbone models fine-tuned with GRPO on the same DeepScaleR-Preview-Dataset using standard discrete tokens. This serves as the direct "same training, different representation" baseline — it uses identical data, optimizer, hyperparameters, and training steps as Multiplex Thinking, differing only in whether tokens are discrete or multiplex.
- Stochastic Soft Thinking (Wu et al., 2025): a recent training-free continuous reasoning baseline that extends the original deterministic Soft Thinking (Zhang et al., 2025) by injecting stochasticity via the Gumbel-Softmax trick. This is the primary continuous reasoning competitor.
- Multiplex Thinking-I: an inference-only variant of the proposed method, using multiplex tokens without any RL training — included in Section 5.1 to isolate the representation's intrinsic benefits from optimization gains.
Generation budget / compute accounting. During training, 8 rollout samples are generated per question with temperature 1.0 and top-p 1.0. During Pass@1 evaluation, top-p is set to 0.95 and results are averaged over 64 runs. During Pass@k evaluation, top-p is set to 1.0 to maximize exploration diversity. A maximum response length of 4,096 tokens is enforced for all stages (training and evaluation), covering both the multiplex thinking phase and the discrete answer phase. For the length scaling experiment in Section 5.3, the discrete CoT baseline is also tested with a 5,120-token budget as a compute-matched comparator. The paper treats one forward pass producing K samples from the same logits distribution as having negligible additional cost versus one forward pass producing one sample, since the K samples share the same logits computation.
Cross-validation / statistical protocol. The Pass@1 metric is averaged over 64 independent runs to provide stable estimates, though the paper does not report standard deviations or confidence intervals. Pass@k curves use bootstrap resampling (1,000 iterations) over 1,024 total rollouts. Model validation during training uses Pass@4 on MATH-500 every 25 steps. No explicit cross-validation split of the test benchmarks is performed — all six evaluation datasets are treated as held-out test sets not used during training.

Main Quantitative Results

Pass@1: Multiplex Thinking Outperforms All Baselines at Both Scales

Table 1 presents the headline Pass@1 results across all six benchmarks and both model scales. Multiplex Thinking achieves the best performance in 11 out of 12 settings (6 datasets × 2 model sizes), with the sole exception being AIME 2025 on the 1.5B model where Stochastic Soft Thinking edges ahead by 0.4 percentage points (13.2 vs. 12.8).

On the 1.5B backbone (top block of Table 1):

Multiplex Thinking surpasses Discrete RL — which shares the identical GRPO training setup — on all six datasets. The margins range from +0.3 on AMC 2023 (38.7 vs. 38.4) and MATH-500 (67.5 vs. 66.7) to +1.9 on Minerva (26.2 vs. 24.3) and +1.3 on AIME 2024 (11.8 vs. 10.5). Since the only difference between these two methods is the token representation during training, these gains directly measure the value of multiplex exploration over discrete sampling under otherwise identical optimization.
Compared to Stochastic Soft Thinking, Multiplex Thinking wins on 4 of 6 benchmarks (AIME 2024: 11.8 vs. 11.2; AMC 2023: tied at 38.7; MATH-500: 67.5 vs. 66.8; Minerva: 26.2 vs. 25.4; OlympiadBench: 31.3 vs. 30.6). The gains are modest but consistent, and significant because Stochastic Soft Thinking is a strong training-free baseline — Multiplex Thinking with RL training beats it on most datasets but not all, suggesting the 1.5B model's capacity may limit how much multiplex exploration can be leveraged.

On the 7B backbone (bottom block of Table 1), the pattern strengthens substantially:

Multiplex Thinking achieves dominant first-place on all six benchmarks. The margins over the next-best method (which varies by dataset) are: AIME 2024: 20.6 vs. 20.3 (Stochastic Soft Thinking); AIME 2025: 19.7 vs. 19.1 (Stochastic Soft Thinking); AMC 2023: 50.7 vs. 47.9 (Stochastic Soft Thinking); MATH-500: 78.0 vs. 76.5 (Stochastic Soft Thinking); Minerva: 38.6 vs. 37.2 (Stochastic Soft Thinking); OlympiadBench: 41.7 vs. 40.6 (Stochastic Soft Thinking).
The margin over Discrete RL is substantially larger at 7B than at 1.5B: +3.4 on AIME 2024 (20.6 vs. 17.2), +6.0 on AMC 2023 (50.7 vs. 44.7), +3.9 on MATH-500 (78.0 vs. 74.1), +3.3 on Minerva (38.6 vs. 35.3), +3.7 on OlympiadBench (41.7 vs. 38.0). This interaction with scale is one of the paper's most important findings: the benefits of multiplex thinking compound with model capacity, likely because larger models can better resolve the interference between superposed reasoning paths embedded in the multiplex token.

The comparison between Multiplex Thinking-I and the training-free baselines (Table 2, 7B only) further clarifies where the gains originate. Multiplex Thinking-I (inference-only, no training) achieves 20.5 on AIME 2024, 19.6 on AIME 2025, 48.6 on AMC 2023, 76.4 on MATH-500, 37.1 on Minerva, and 40.6 on OlympiadBench — competitive with or exceeding Stochastic Soft Thinking on 4 of 6 datasets. The full trained Multiplex Thinking then adds further gains (20.6, 19.7, 50.7, 78.0, 38.6, 41.7), confirming that (a) multiplex representations provide intrinsic benefits before optimization, and (b) RL training amplifies these benefits.

Pass@k Scaling: Multiplex Thinking Widens the Gap at Larger Sampling Budgets

Figure 2 (with the complete results across all six datasets in Appendix Figure 7) examines how performance scales with test-time sampling budget from Pass@1 to Pass@1024. This analysis probes the exploration upper bound — how many correct solutions exist in the model's output distribution, and how effectively different methods can surface them given more samples.

The headline finding is that Multiplex Thinking achieves a higher and more steeply scaling Pass@k curve than discrete baselines, with the gap widening as k increases on harder problems. Specific patterns:

On AIME 2025 (7B), the starkest gap: the Discrete RL baseline plateaus around 40% at k=1024, while Multiplex Thinking continues to scale, reaching approximately 55%. This ~15 percentage point gap at large k represents a substantial expansion of the model's effective exploration space — multiplex tokens preserve alternative reasoning paths that discrete decoding abandons, and those preserved paths contain correct solutions that would otherwise be inaccessible.
On AIME 2024 (7B), a similar widening pattern: Discrete RL reaches roughly 42–44% at k=1024, while Multiplex Thinking reaches approximately 50–52%. The gap is smaller than on AIME 2025 but still substantial (~8–10 points).
On OlympiadBench (7B), the gap expands with k: both methods start close at low k but Multiplex Thinking pulls ahead as k grows, reaching approximately 72% vs. 65% at k=1024 (estimated from Figure 7).
On MATH-500, saturation effects dominate: all methods approach ceiling performance (90%+) at high k, so the gap compresses. This is expected — MATH-500 is the easiest benchmark, and once accuracy nears 100%, there is no room for differences to manifest.
On the 1.5B backbone: the patterns are qualitatively similar but compressed. Multiplex Thinking maintains an advantage over discrete baselines across most k values, but the absolute differences are smaller, consistent with the Pass@1 finding that smaller models benefit less from multiplex exploration.

The paper interprets these curves as evidence for three claims: (1) multiplex thinking expands the intrinsic exploration capacity of the model — there exist correct solutions in the multiplex distribution that are effectively unreachable via discrete sampling; (2) the benefits are difficulty-dependent, with the largest gains on the hardest problems where reasoning traces contain many uncertain decision points; (3) multiplex thinking provides superior sample efficiency — to achieve a target accuracy, fewer multiplex rollouts are needed than discrete rollouts.

Length Scaling: Multiplex Thinking Achieves Higher Accuracy with Shorter Sequences

Figure 4 and Table 3 examine the trade-off between sequence length and accuracy, comparing Multiplex Thinking-I (inference-only, no training) against Discrete CoT at different token budgets.

The headline finding from Table 3: Multiplex Thinking-I with a 4,096-token budget (40.5% average accuracy across six benchmarks) outperforms Discrete CoT with a 5,120-token budget (39.6%), a 25% larger budget. The bracketed annotations in the table emphasize the contrast: Discrete CoT improves by 2.8 points when increasing from 4k to 5k tokens (35.8 → 39.6), but Multiplex Thinking-I-4k jumps ahead by 4.7 points relative to Discrete CoT-4k (35.8 → 40.5).

Figure 4 shows this as a function of sequence length: the Multiplex Thinking-I curve lies above the Discrete CoT curve at every length tested, with the gap appearing roughly constant in absolute terms. This demonstrates that multiplex thinking's efficiency advantage is not merely about achieving the same accuracy with fewer tokens — it achieves higher accuracy at every length budget, and the improvement cannot be replicated by simply giving the discrete baseline more tokens.

The training dynamics in Figure 5 reinforce this point: over the course of GRPO training, Multiplex Thinking trajectories are consistently shorter than Discrete RL trajectories (the multiplex response length curve runs below the discrete curve throughout training), yet Multiplex Thinking achieves higher accuracy. This is direct evidence that multiplex tokens carry higher "information density" — each multiplex token encodes what would require multiple discrete tokens to express, compressing the reasoning process into fewer steps.

Ablation Studies and Robustness Checks

Multiplex width K (Section 5.2, Figure 3): The transition from K=1 (discrete RL) to K=2 produces the largest accuracy jump across all benchmarks — e.g., on AMC 2023 (7B), from 44.7% to 49.6% (+4.9 points). Further increases to K=3 and K=6 yield diminishing returns, with K=3 and K=6 performance often nearly overlapping. On some benchmarks (AIME 2024, AIME 2025), K=2, 3, and 6 are essentially tied. The paper interprets this as evidence that the primary gain is from breaking the single-token bottleneck — adding even one extra alternative captures the most critical exploration benefit, while additional samples add primarily lower-probability tokens that contribute marginal value. The Pass@k extension in Figure 8 (Appendix A.2.2) confirms this pattern: K≥2 curves cluster together well above the K=1 baseline across all k.
Token aggregation strategy (Section 5.5, Table 5): Comparing LM-head reweighting (Multiplex Thinking-Weighted) against simple uniform averaging of embeddings (Multiplex Thinking-Averaged) reveals comparable performance across both model scales and all six benchmarks. On the 7B backbone, the differences are small and not consistently favoring either method: Weighted wins on AIME 2024 (20.6 vs. 19.9), AMC 2023 (50.7 vs. 49.9), MATH-500 (78.0 vs. 77.6), and Minerva (38.6 vs. 38.4); Averaged wins on AIME 2025 (20.0 vs. 19.7) and OlympiadBench (41.8 vs. 41.7). Both variants substantially outperform the Discrete RL baseline (Table 5 bottom block: 17.2, 17.1, 44.7, 74.1, 35.3, 38.0). The paper concludes that "the effectiveness of Multiplex Thinking stems from the inclusion of diverse reasoning paths in the latent space rather than the specific weighting scheme" — a robustness result that suggests the model's transformer layers can extract relevant features from the multiplex representation regardless of the precise linear combination coefficients.
Inference-only vs. trained multiplex (Section 5.1, Table 2): Evaluating Multiplex Thinking-I (no RL training) against Stochastic Soft Thinking and Discrete CoT on the 7B backbone tests whether gains come from the representation itself or from better RL optimization. Multiplex Thinking-I outscores Stochastic Soft Thinking on 4 of 6 datasets (AIME 2024: 20.5 vs. 20.3; AIME 2025: 19.6 vs. 19.1; AMC 2023: 48.6 vs. 47.9; OlympiadBench: tied at 40.6) and substantially outperforms Discrete CoT across the board. Adding RL training (Multiplex Thinking row) then produces additional gains of 0.1–2.1 points over Multiplex Thinking-I. This establishes a two-part finding: (1) multiplex representations are intrinsically beneficial, (2) RL amplifies the benefit.
Entropy collapse during training (Section 5.4, Table 4): The entropy reduction ratio — the relative decrease in average policy entropy from the first 10 training steps to the last 10 — is lower for multiplex training (5.82% for K=2, 6.03% for K=3, 7.09% for K=6) than for discrete RL (9.44% for K=1). A smaller reduction indicates less entropy collapse, meaning the policy retains more exploration diversity throughout training. This provides a mechanistic explanation for the improved Pass@k scaling: multiplex training prevents the policy from prematurely collapsing onto a narrow set of behaviors, preserving the diversity needed to discover hard-to-find correct solutions at test time. The non-monotonic pattern (entropy reduction at K=3 is slightly higher than at K=2) is not discussed but may reflect noise in the measurement.
Stopping criteria design (Appendix A.1): The paper reports that training-free heuristics for early stopping — such as monitoring consecutive low-entropy tokens — were attempted but found to "introduce artificial patterns that the model tends to exploit during RL optimization, resulting in training instability and the generation of incoherent content." The adopted approach — triggering the end of thinking when the highest-probability token is [eot] — avoids reward hacking because the model learns to produce [eot] at appropriate times through the RL reward signal itself. This is framed as a design lesson rather than a formal ablation, but it represents an important robustness consideration: handcrafted control mechanisms create exploitable patterns that RL will find and exploit.
Pass@k under different widths (Figure 8, Appendix A.2.2): Extending the K-size ablation to full Pass@k curves confirms the Section 5.2 finding: there is a large gap between K=1 and K≥2 across all k values, but the K=2, 3, and 6 curves are closely clustered. This suggests that the exploration advantage of multiplex thinking (as distinct from the accuracy advantage at Pass@1) is primarily driven by the mere presence of multiple samples rather than their number — having any diversity at each step is enough to significantly expand the reachable solution space.

Critical Assessment

Claim: Multiplex thinking consistently outperforms discrete CoT and RL baselines at both Pass@1 and Pass@k.

The evidence for this claim is strong within the tested conditions. Table 1 shows wins in 11 of 12 settings, Figure 2 shows widening Pass@k advantages on harder benchmarks, and the gains persist across both model scales. However, several caveats apply:

The absolute margins are modest on some benchmarks. On AIME 2024 (1.5B), Multiplex Thinking scores 11.8 vs. 11.2 for Stochastic Soft Thinking and 10.5 for Discrete RL — a 1.3-point spread among three methods on a benchmark where all scores are low. Statistical significance is not reported, and with only 64 evaluation runs, the variance could be non-trivial. The consistency of the ranking across datasets provides some reassurance, but confidence intervals would substantially strengthen the claim.
The comparison between Multiplex Thinking and Stochastic Soft Thinking conflates two variables: training vs. no training. Stochastic Soft Thinking is a pure inference-time method; Multiplex Thinking is trained with GRPO. The fairer comparison for "representation quality" is Multiplex Thinking-I vs. Stochastic Soft Thinking (Table 2), where the margins are razor-thin (within 0.2–0.5 points on most benchmarks). This suggests that the primary advantage of multiplex thinking over prior continuous methods is its RL compatibility, not a dramatic improvement in the quality of continuous representations per se. The trained version does outperform Stochastic Soft Thinking more convincingly, but this advantage is attributable to the combination of representation + optimization, not the representation alone.

Claim: The gains stem from the multiplex representation rather than the RL training process alone.

This is well-supported by the Discrete RL comparison. Multiplex Thinking and Discrete RL share identical training data, algorithm (GRPO), hyperparameters (Table 6), and training steps. The only difference is token representation during thinking. The consistent superiority of the multiplex variant therefore isolates the representation's contribution. However, the lack of a fully matched "same training with continuous deterministic tokens" baseline (e.g., training Soft Thinking with GRPO) prevents determining whether the advantage comes specifically from stochastic multiplexing or from continuous representations in general. COCONUT-style hidden-state tokens were not tested.

Claim: Multiplex thinking improves token efficiency — higher accuracy with shorter sequences.

Figure 5 shows that Multiplex Thinking trains to shorter average response lengths than Discrete RL while achieving higher accuracy. Table 3 shows that Multiplex Thinking-I-4k outperforms Discrete CoT-5k. But the token efficiency claim requires careful interpretation: during the thinking phase, a multiplex token with K=3 requires sampling 3 discrete tokens and averaging their embeddings — the "token" fed forward is still one embedding vector, but it was constructed from 3 samples. Whether this should count as 1 token or 3 tokens for efficiency purposes depends on the cost model. If the dominant cost is the transformer forward pass (which processes one embedding per step regardless of K), then multiplex thinking genuinely saves compute. If sampling overhead matters, the savings are less clear. The paper does not report wall-clock time or FLOPs comparisons.

Claim: Benefits are difficulty-dependent, with larger gains on harder problems.

The Pass@k curves (Figure 2) support this: the gap between multiplex and discrete widens on AIME 2024/2025 (hard) but compresses on MATH-500 (easier, due to ceiling effects). However, the paper does not provide a systematic difficulty-bin analysis (in contrast to the reference example paper's five-quintile breakdown), which would have strengthened this claim considerably. The per-dataset results in Table 1 do not show a clean correlation between dataset difficulty and multiplex margin: the margin over Discrete RL is larger on AMC 2023 (+6.0 at 7B) than on the harder AIME 2024 (+3.4) or AIME 2025 (+2.6). The Pass@k widening-gap narrative holds for some datasets (AIME 2025) but the difficulty story is more nuanced than the paper's framing suggests.

Genuine weaknesses:

Single model family, single domain. All results are on DeepSeek-R1-Distill-Qwen on math reasoning benchmarks. Whether multiplex thinking transfers to non-distilled base models, other architectures (Llama, Mistral), or other reasoning domains (code, logic, science) is untested. The choice of an already reasoning-tuned backbone (distilled from R1) means the models already have strong CoT capabilities — whether multiplex thinking helps weaker base models is unknown.
No FLOPs-matched or wall-clock comparison. The paper emphasizes efficiency but never reports actual compute costs (FLOPs, GPU hours, latency). The statement that "increasing K does not require additional forward passes beyond sampling from the same logits distribution" is true for the forward pass but ignores the embedding lookup and averaging operations, which are negligible in practice but should be quantified for a complete efficiency analysis.
The GRPO configuration lacks standard regularization. Training with zero KL penalty and zero entropy penalty for 300 steps without a reference model is unusual in RLHF/RLVR practice. The paper justifies this by arguing that multiplex tokens maintain sufficient exploration on their own (supported by the entropy analysis), but no ablation tests whether adding a small KL penalty would improve or harm performance. The possibility that multiplex thinking is simply more robust to aggressive optimization (rather than intrinsically better) is not disentangled.
The inference-only variant (Multiplex Thinking-I) is a crucial diagnostic but receives minimal analysis. Table 2 shows it is competitive with Stochastic Soft Thinking, but there is no analysis of why — does it help uniformly across problems, or only on certain types? Does it interact with problem difficulty the same way the trained version does? A difficulty-bin analysis of Multiplex Thinking-I would have illuminated whether the representation's intrinsic benefits are difficulty-dependent (as the Pass@k curves suggest for the trained version) or uniform.
No combination with outer-loop parallel reasoning. The paper positions multiplex thinking as "complementary to existing parallel reasoning strategies" and suggests combining it with Self-Consistency or Best-of-N. This combination is never tested. A simple experiment — run Self-Consistency over N multiplex trajectories vs. N discrete trajectories — would have directly tested whether the per-step exploration advantage compounds with trajectory-level diversity, or whether the two forms of exploration are partially redundant.
Small evaluation set sizes. The six benchmarks vary in size (MATH-500 has 500 problems; AIME 2024 and 2025 have 30 each), but no breakdown is provided. The smaller benchmarks (AIME) are where the most interesting scaling behavior occurs (widening Pass@k gaps), and results on 30 problems with bootstrap resampling may have high variance not captured by the current reporting.
Training data overlap concerns. The training set (DeepScaleR-Preview-Dataset, ~40K problems) may overlap with the evaluation benchmarks' problem sources. The paper does not discuss decontamination, and overlap between training and evaluation would inflate Pass@1 numbers for all trained methods (including Discrete RL). The gap between trained and untrained methods (Table 2) partially addresses this by showing that Multiplex Thinking-I also improves, but a proper decontamination analysis would strengthen confidence.

Missing experiments that would have strengthened the paper:

A continuous-but-deterministic RL baseline: train the original Soft Thinking (probability-weighted embedding mixture) with the same GRPO setup. This would test whether the stochastic sampling is necessary for RL optimization, or whether any continuous representation (even deterministic) provides similar benefits when trained.
A difficulty-stratified analysis: break down results by problem difficulty (similar to the five-quintile approach in the reference paper) to test whether multiplex thinking's benefits are uniform or concentrated on problems with specific characteristics (e.g., those requiring many decision points, those with high per-step entropy).
A wall-clock time and memory comparison: report actual inference latency and GPU memory usage for K=1, 2, 3, 6 to ground the efficiency claims in practical terms.
A multiplex + self-consistency experiment: compare Discrete CoT with majority voting over N samples vs. Multiplex Thinking with majority voting over N multiplex trajectories, to test complementarity.
Testing on a non-reasoning-tuned base model: the gains on DeepSeek-R1-Distill-Qwen may partly reflect that distilled models already have structured internal representations that multiplexing can exploit. Testing on a base pretrained model (without distillation) would test generality.

6. Limitations and Trade-offs

6.1 Difficulty Estimation Cost Is Unaccounted for — Difficulty Bins Are Not Used

The reference paper example demonstrates a compute-optimal framework whose central practical bottleneck is the cost of estimating prompt difficulty before allocating the inference budget — generating 2048 samples per question to bin by pass@1, then selecting per-bin strategies. Multiplex Thinking faces a structurally analogous but more subtle version of this problem. The method is self-adaptive: the multiplex token automatically collapses to discrete-like behavior on low-entropy (confident) steps and expands to continuous exploration on high-entropy (uncertain) steps, driven entirely by the model's own next-token distribution (Section 3.1). This is elegant, but it means the allocation of exploration compute is purely local — each step's K-way branching is determined by that step's entropy, with no global mechanism that assesses whether the overall problem warrants more or less exploration.

The consequence is a misallocation of the implicit compute budget. On easy problems where the model is confident throughout, multiplex thinking behaves nearly identically to discrete CoT (all K samples collapse to the same token, Figure 6 consensus steps), so the extra sampling cost (K× token probabilities computed, K× embeddings averaged) is wasted — it produces no exploration benefit because there is no uncertainty to resolve. On the hardest problems (difficulty bin 5 in the reference paper's taxonomy), the entropy may be high at many steps, but the base model lacks the fundamental capability to produce correct solutions regardless of exploration budget — multiplex thinking expands the search space but the correct solution is not in it. The paper's own data supports both failure modes:

On MATH-500 (the easiest benchmark, 7B model), performance saturates quickly for all methods as accuracy approaches ceiling (Figure 2, bottom row, second column), and the gap between multiplex and discrete compresses — suggesting multiplex exploration provides little marginal value when problems are already solvable.
On AIME 2025 (7B), the Discrete RL baseline plateaus around 40% at k=1024, and multiplex thinking pushes this to ~55% (Figure 2, top row, second column) — a substantial gain, but still only 55%, meaning nearly half the problems remain unsolved regardless of the exploration expansion.

The paper provides no difficulty-stratified analysis — no breakdown of which problems benefit from multiplex thinking and which do not. Unlike the reference paper's five-quintile binning that reveals qualitatively different strategy effects at different difficulty levels, Multiplex Thinking's results are reported only as dataset-level aggregates. The per-benchmark results in Table 1 do not show a clean correlation between benchmark difficulty and multiplex margin: the margin over Discrete RL on AMC 2023 (+6.0 at 7B) is larger than on the harder AIME 2024 (+3.4) or AIME 2025 (+2.6). This suggests that difficulty, as measured by aggregate benchmark pass rates, is not the sole driver of multiplex benefit — problem structure (e.g., number of high-entropy decision points) likely matters, but the paper does not investigate this.

Mitigation status: The paper does not acknowledge this limitation or propose a difficulty estimation mechanism. The self-adaptive per-step confidence gating is treated as a feature, not a partial solution needing complement. A natural extension — estimating problem difficulty from initial token distributions and modulating K or the number of parallel rollouts accordingly — is not discussed.

6.2 Multiplex-Thinking-I Is Competitive with Stochastic Soft Thinking but Lacks an RL-Based Deterministic Continuous Baseline

The paper's central claim is that stochastic continuous tokens enable RL optimization, while deterministic continuous tokens (Soft Thinking, COCONUT) cannot be optimized with likelihood-ratio objectives because they lack a well-defined probability distribution over trajectories (Section 3). This claim is foundational: it is the justification for the entire sampling-based architecture. However, the paper never tests an RL-trained deterministic continuous baseline.

The evidence for the claim is structural (the mathematical argument in Section 3.1) and indirect (Multiplex Thinking outperforms Discrete RL, and Multiplex Thinking-I outperforms the training-free Stochastic Soft Thinking). But a direct test would be to train Soft Thinking — which constructs continuous tokens via $c_i = \sum_{k \in V} p_i(k) e(k)$ (Zhang et al., 2025) — with the same GRPO setup used for multiplex thinking. This is technically challenging because deterministic continuous tokens lack token-level log-probabilities, but one could use the REINFORCE estimator over the answer tokens only, treating the thinking trace as a deterministic prefix. If this baseline performed comparably to Multiplex Thinking, it would undermine the paper's central thesis that stochasticity is necessary. If it performed substantially worse, it would validate the thesis directly.

The absence of this baseline leaves a logical gap. The comparison between Multiplex Thinking and Discrete RL (Table 1) shows that continuous representations help, but it does not show that stochastic continuous representations help more than deterministic continuous representations would, given the same RL training. The comparison between Multiplex Thinking-I and Stochastic Soft Thinking (Table 2, 7B) shows that inference-only multiplex is roughly comparable to inference-only stochastic continuous reasoning (margins of 0.1–0.5 points), which is consistent with either interpretation: (a) stochasticity helps, or (b) any continuous representation that stays in vocabulary embedding space is about equally good at inference time, and the RL advantage of multiplex thinking is separate from its stochasticity.

Evidence in the paper: Table 2 shows Multiplex Thinking-I vs. Stochastic Soft Thinking (both inference-only, 7B): margins are 0.2 (AIME 2024), 0.5 (AIME 2025), 0.7 (AMC 2023), -0.1 (MATH-500), -0.1 (Minerva), 0.0 (OlympiadBench). The trained Multiplex Thinking then adds 0.1–2.1 points over Multiplex Thinking-I. This pattern is consistent with the RL optimization providing a modular gain on top of a representation whose inference-time quality is similar to Stochastic Soft Thinking.

Mitigation status: Not addressed. The paper argues positionally (Section 1, Section 3.1) that deterministic continuous tokens are "fundamentally misaligned with RL" but does not test this claim against an empirical counterfactual. The authors may have considered this infeasible due to the lack of token-level log-probabilities in deterministic methods, but the paper does not discuss the omission.

6.3 Single Model Family, Single Domain — Generality Is Unestablished

All experiments use DeepSeek-R1-Distill-Qwen at 1.5B and 7B scales, evaluated exclusively on math reasoning benchmarks (AIME 2024/2025, AMC 2023, MATH-500, Minerva Math, OlympiadBench). The choice of backbone is significant: these are models that have already been distilled from a larger R1-style reasoning model, meaning they have undergone specialized training to produce structured chain-of-thought reasoning. The paper does not test on a base pretrained model without reasoning-specific fine-tuning, nor on other model families (Llama, Mistral, Gemma), nor on other reasoning domains (code generation, logical reasoning, scientific QA, multi-hop question answering).

The consequence is that we cannot distinguish between two hypotheses:

Multiplex thinking helps generally — the representation's information density and exploration properties benefit any transformer-based LLM on any reasoning task.
Multiplex thinking helps specifically for distilled reasoning models on math — the structured CoT traces produced by R1-distilled models create the right conditions (frequent high-entropy decision points, clear separation between thinking and answer phases, well-defined vocabulary embedding spaces after distillation) for multiplex exploration to be productive.

Several aspects of the results suggest Hypothesis 2 may have substantial explanatory power. The interaction with scale (Section 5, Table 1: margins over Discrete RL are consistently larger at 7B than at 1.5B) suggests that multiplex thinking benefits from model capacity — larger models can better resolve the interference between superposed reasoning paths. This could mean the method underperforms on smaller or weaker models where the transformer cannot effectively process off-manifold embedding mixtures. If true, this would limit applicability to the relatively large, capable models used in the paper.

The domain restriction to math is also consequential. Math reasoning has clean correctness signals (answers are right or wrong, no ambiguity), well-defined intermediate reasoning steps (algebraic manipulations, theorem applications), and is the domain where CoT reasoning is most studied and most effective. Whether multiplex thinking transfers to domains with fuzzier reasoning structure — explaining a concept, writing an argument, debugging code — is unknown. The paper's qualitative example (Figure 6) shows that multiplex exploration steps cluster at specific lexical decision points ("first" vs. "different" vs. "car"), which may be more characteristic of math reasoning's structured, formulaic language than of more open-ended generation.

Evidence in the paper: All experiments (Tables 1–5, Figures 2–9) use the same two backbone models on the same six math benchmarks. The paper does not discuss domain or model family limitations.

Mitigation status: Not addressed. The paper does not claim generality beyond math reasoning, but the framing (Section 1: "a stochastic soft reasoning mechanism") and the related work positioning (bridging discrete CoT and continuous reasoning broadly) imply broader applicability. Future work on other domains and model families is not explicitly suggested.

6.4 No FLOPs, Memory, or Latency Accounting — The Efficiency Claim Is Unsubstantiated

The paper makes a prominent efficiency claim: multiplex thinking produces "shorter sequences" (Section 5.3), achieves "improved token efficiency" (Section 7 Conclusion), and "does not require additional forward passes beyond sampling from the same logits distribution" (Section 5.3). These claims are used to argue that multiplex thinking is not just more accurate but also more compute-efficient than discrete baselines.

However, the paper never reports wall-clock time, FLOPs, GPU memory usage, or any hardware-level cost metric. The only cost proxy is sequence length (number of tokens), and even this is incomplete: during the thinking phase, a multiplex token with K=3 requires (a) computing the next-token distribution (one forward pass, same as discrete), (b) drawing K independent samples (negligible compute), (c) looking up K token embeddings in the embedding matrix, (d) computing the weighted average of K d-dimensional vectors (negligible compute for small K, typical d), and (e) feeding the resulting continuous vector as input to the next step (one embedding, same as discrete). The forward pass dominates, so steps (a) and (e) are the relevant costs, and they are identical to discrete generation in terms of forward passes per step.

But the comparison is not "multiplex step vs. discrete step" — it is "entire multiplex trajectory vs. entire discrete trajectory." Figure 5 shows that multiplex trajectories are shorter on average than discrete trajectories during training, which the paper interprets as evidence of efficiency. However, the sequence length metric counts both thinking and answer tokens. A shorter multiplex trajectory might use fewer steps but each step processes a multiplex token constructed from K samples. If the dominant cost is forward passes per reasoning step, then multiplex thinking is more efficient only if the reduction in number of steps outweighs any per-step overhead multiplied by K. The paper provides no data on this trade-off.

Furthermore, the training cost is unaccounted for. Multiplex thinking requires GRPO training (300 steps, 8 rollouts per question, batch size 128), which is identical in setup to Discrete RL training but may have different convergence properties and computational requirements. The paper does not compare training FLOPs, training wall-clock time, or memory usage between multiplex and discrete training. If multiplex training requires more GPU memory (e.g., because the continuous embeddings have different numerical properties, or because the K per-step log-probabilities increase the computation graph size), then the training efficiency advantage may be smaller or negative.

Evidence in the paper: Figure 5 (response length dynamics) and Table 3 (length scaling) use token count as the sole efficiency metric. Figure 4 plots accuracy vs. response length. No FLOPs, latency, or memory data is reported.

Mitigation status: The paper does not acknowledge this as a limitation. The efficiency claim is stated as a conclusion ("shorter sequences" → "better token efficiency") without quantifying what "efficiency" means in practice. The Appendix A.1 notes experiments use 8× NVIDIA DGX B200 GPUs with bfloat16 precision, but this is hardware specification, not performance measurement.

6.5 The Transition from Thinking to Answering Uses an Argmax Rule That RL Could Exploit

The paper uses a specific mechanism to decide when to stop multiplex thinking and begin answer generation: at each thinking step, the system checks whether $\arg\max_v p_i(v)$ equals the [eot] token ID; if so, the thinking phase ends and discrete answer generation begins (Appendix A.1). This criterion is applied deterministically — it does not depend on the K samples drawn at that step, only on the raw next-token distribution before sampling.

The paper defends this choice by noting that training-free heuristics (e.g., monitoring for consecutive low-entropy tokens) "introduce artificial patterns that the model tends to exploit during RL optimization, resulting in training instability and the generation of incoherent content." The implicit claim is that the argmax rule is robust to such exploitation because the model learns to produce [eot] at appropriate times through the reward signal: thinking indefinitely without producing an answer yields zero reward, so the model is incentivized to produce [eot] when further thinking is unlikely to improve the answer.

However, the argmax rule introduces its own exploitable structure. The rule is: whenever the model's peak next-token probability is on [eot], regardless of how uncertain the distribution actually is, stop thinking. This creates an incentive to produce distributions where [eot] has the highest individual probability — not necessarily because the model is done reasoning, but because placing peak mass on [eot] terminates the thinking phase and allows the model to produce an answer (which may or may not be correct, but which at least yields a reward signal). The model could learn to spike the probability of [eot] while keeping the distribution over other tokens diffuse (high entropy overall, but with [eot] as the single highest-probability token), effectively gaming the stopping rule to end thinking prematurely — or to continue thinking indefinitely by avoiding placing peak mass on [eot] even when reasoning is complete, exploring more to increase the chance of stumbling onto a correct path. Both failure modes are consistent with an RL objective that rewards only the final answer: the model has no direct incentive to produce [eot] at the "right" time, only to produce it at a time that maximizes expected reward.

The paper does not investigate whether this exploitation occurs. There is no analysis of thinking-phase length distributions (e.g., do multiplex trajectories sometimes end suspiciously early or continue suspiciously long compared to discrete baselines?), no comparison to alternative stopping rules (e.g., sampling-based: stop when [eot] is one of the K samples; or confidence-based: stop when the entropy over non-[eot] tokens falls below a threshold), and no measurement of whether the [eot] argmax correlates with actual reasoning completion or is simply learned as a reward-maximizing trigger.

Evidence in the paper: Appendix A.1 describes the stopping criteria and acknowledges the issue with heuristic-based rules but does not test whether the argmax rule is also vulnerable. No ablation on stopping criteria is reported.

Mitigation status: Partially addressed in principle (the paper acknowledges that heuristics can be exploited) but not evaluated for the specific argmax rule used. The claim that "the RL objective naturally regulates the thinking process" is asserted, not demonstrated.

6.6 Small Evaluation Sets on the Hardest Benchmarks Where the Most Interesting Claims Are Made

The paper's most striking finding — that multiplex thinking maintains a widening Pass@k gap over discrete baselines at large k on hard problems — is concentrated on the smallest evaluation benchmarks. AIME 2024 and AIME 2025 each contain 30 problems (standard AIME format: 15 problems per exam, two exams per year). OlympiadBench is larger but its exact size is not specified in the paper (the reference He et al., 2024 describes it as containing "Olympiad-level bilingual multimodal scientific problems"). AMC 2023 is similarly a competition exam with a limited number of problems. MATH-500 has 500 problems and Minerva Math has several hundred.

The consequence is that the Pass@k curves for AIME (Figure 2, first two columns) — which show the most dramatic widening gaps and are the paper's primary evidence for the exploration expansion claim — are estimated from 30 data points. The bootstrap procedure (1,000 iterations over 1,024 runs per problem) provides within-problem variance estimates but cannot compensate for the small number of independent problems. A single problem where multiplex thinking discovers a correct solution that discrete CoT misses could shift the Pass@1024 estimate by several percentage points. The paper does not report confidence intervals on the Pass@k curves, making it impossible to assess whether the ~15 point gap on AIME 2025 (7B, k=1024) is statistically distinguishable from, say, a 5-point gap.

This is especially concerning because the characteristics that make AIME problems hard — multi-step reasoning, non-obvious solution paths, high branching factors — are precisely the conditions where multiplex thinking's exploration advantage should be most pronounced (per the entropy analysis in Section 3.3). If the AIME results are noisy due to small sample size, the paper's central claim about exploration scaling may be weaker than it appears. The MATH-500 results, where sample size is adequate (500 problems), show a much smaller and compressing gap — consistent with the ceiling-effect interpretation, but also consistent with the possibility that multiplex thinking's exploration advantage is modest on larger, more statistically stable benchmarks.

Evidence in the paper: Benchmark sizes are not reported in the main text. AIME is described as a competition benchmark; the 2024 and 2025 versions are standard 30-problem sets. Pass@k curves use bootstrap resampling (1,000 iterations, 1,024 total runs per problem) as described in Section 4.1, but no confidence bands are shown in Figure 2 or Figure 7.

Mitigation status: Not addressed. The paper does not discuss benchmark sizes or statistical power. Future work on larger evaluation sets with more stable high-k estimates is not suggested.

7. Implications and Future Directions

How This Work Changes the Landscape

This paper introduces a new axis for scaling test-time compute — multiplex width K — that is orthogonal to both sequence length and outer-loop parallel sampling (self-consistency, Best-of-N). Before this work, the field's understanding of how to allocate inference compute was roughly two-dimensional: you could increase the number of reasoning steps (longer chains of thought) or increase the number of independent trajectories (more parallel samples). Multiplex Thinking adds a third dimension: per-step exploration breadth, where each reasoning step can simultaneously consider multiple alternatives encoded into a single continuous token, trading off within-step diversity against step count. This is not merely an efficiency trick — it reorganizes how exploration happens during both training and inference, shifting from depth-first trajectory search (each rollout commits to a single discrete path) toward breadth-first local search (each step retains multiple plausible continuations in superposition).

The magnitude of this shift is best characterized as a reframing that opens a previously closed door rather than a paradigm overthrow of discrete CoT. The paper does not argue that discrete reasoning is obsolete — the answer phase still uses standard discrete tokens, and K=1 recovers the discrete baseline exactly. Rather, it demonstrates that the space between "fully discrete" and "fully deterministic continuous" reasoning contains useful intermediate points that prior work had dismissed or not recognized. Soft Thinking (Zhang et al., 2025) collapsed the next-token distribution to a single expectation; COCONUT (Hao et al., 2025) collapsed it to a hidden state. Both are deterministic endpoints. Multiplex Thinking shows that stochastic continuous tokens — tokens constructed from discrete samples rather than deterministic functions of logits — are both practically feasible and RL-compatible, occupying a design point that combines the exploration benefits of discrete sampling with the information density of continuous representations.

This reframing has two downstream consequences for the research landscape:

First, it makes RL-based optimization of continuous reasoning tractable. The paper's most immediate practical contribution is showing that a standard GRPO training loop — with zero modifications to the algorithm, hyperparameters, or reward structure — can directly optimize models that use continuous reasoning tokens, as long as those tokens are constructed from discrete samples whose log-probabilities are available. This removes the primary barrier that had kept continuous reasoning methods in the training-free or special-purpose-training regime. Before this work, if you wanted to train a model to reason in a continuous latent space, you had to design a custom training objective (e.g., COCONUT's multi-stage curriculum with distillation losses). After this work, you can just run GRPO. This makes continuous reasoning accessible to the broader RL-for-reasoning community without requiring specialized infrastructure or loss functions.

Second, it reveals that the key bottleneck in continuous reasoning is not representational but stochastic. The paper's comparison between Multiplex Thinking-I (inference-only) and Stochastic Soft Thinking (Table 2) shows that two very different methods for constructing stochastic continuous tokens — independent sampling and averaging vs. Gumbel-Softmax injection — achieve nearly identical inference-time performance. The trained Multiplex Thinking then substantially exceeds both. This pattern strongly suggests that the primary value of continuous tokens is in enabling better RL exploration during training, not in providing intrinsically better representations at inference time. The inference-only gains are real but small (roughly 0.5–5 points over Discrete CoT, depending on benchmark); the training-time gains compound on top of this (roughly another 1–4 points over the inference-only version). This shifts the research question from "how do we design better continuous token representations?" toward "how do we design continuous representations that maximize exploration diversity during RL?" — a fundamentally different optimization target.

The paper also provides a clean resolution to a tension in the continuous reasoning literature. Prior work had established that continuous tokens can help (Soft Thinking improves over discrete CoT at inference time) but had not shown they can be trained with RL (deterministic continuous tokens crash or drift under policy gradient optimization). The pessimistic interpretation — "continuous reasoning is fundamentally incompatible with RL" — turns out to be false, but only when the continuous tokens preserve stochasticity through discrete sampling. The paper thus reconciles the apparent contradiction: continuous tokens help, RL helps, and they can be combined, but only if the continuous token construction respects the probabilistic semantics that RL objectives require.

Follow-Up Research This Work Enables

Training Soft Thinking with GRPO as a deterministic continuous RL baseline. The paper's central claim is that stochasticity is necessary for RL optimization of continuous reasoning — deterministic methods like Soft Thinking lack token-level log-probabilities and therefore cannot be optimized with likelihood-ratio objectives (Section 3.1). This claim is argued structurally but never tested empirically. A strong follow-up would train the original Soft Thinking — which constructs continuous tokens as $c_i = \sum_{v} p_i(v) e(v)$ , a deterministic probability-weighted embedding mixture — using the same GRPO setup as Multiplex Thinking. The challenge is that Soft Thinking trajectories have no well-defined log-probability, but one could use the REINFORCE estimator over only the answer tokens, treating the thinking trace as a deterministic prefix that influences the answer distribution without itself being optimized. If this baseline underperforms Discrete RL (because the thinking trace is a fixed function of the context that RL cannot improve), it validates the necessity of stochasticity. If it performs comparably to Multiplex Thinking, it would undermine the paper's core thesis and suggest that the gains come from continuous representations per se rather than their stochastic construction. The experiment requires implementing a custom GRPO variant that backpropagates through the deterministic thinking trace to the answer logits, which is non-trivial but feasible using modern automatic differentiation frameworks.

Difficulty-stratified analysis of multiplex benefit with per-problem entropy profiling. The paper shows that multiplex gains vary across benchmarks (larger on AMC 2023 +6.0 points at 7B than on AIME 2024 +3.4 points at 7B, Table 1), but provides no systematic analysis of which problems benefit and why. A strong follow-up would bin evaluation problems by their average per-step entropy under the discrete CoT baseline and measure multiplex thinking's accuracy gain per bin. The hypothesis — motivated by the entropy analysis in Section 3.3 and the qualitative visualization in Figure 6 — is that multiplex thinking helps most on problems with many high-entropy decision points (where the single-token bottleneck is most damaging) and helps least on problems that are either trivially easy (low entropy throughout, all steps are confident) or impossibly hard (the model has no correct paths in its distribution regardless of exploration). This would produce a difficulty-gain curve analogous to the reference paper's five-quintile analysis, revealing whether multiplex thinking's benefit is concentrated on a specific difficulty regime. The experiment requires: (1) running the discrete CoT baseline on each evaluation problem and logging the next-token entropy at each step, (2) computing per-problem average entropy (or entropy above some threshold), (3) binning problems into quantiles, (4) measuring multiplex vs. discrete accuracy per bin. This would also directly test whether the self-adaptive per-step gating (which modulates exploration step-by-step but not problem-by-problem) is sufficient or whether a global difficulty estimate would further improve allocation.

Multiplex thinking on non-distilled base models to test generality of the representation benefit. All experiments use DeepSeek-R1-Distill-Qwen, a model already fine-tuned for structured chain-of-thought reasoning through distillation from a larger R1 model. The structured, formulaic CoT traces produced by distilled models may be particularly amenable to multiplexing — the reasoning steps follow predictable patterns where high-entropy decision points are lexically identifiable ("first" vs. "different" vs. "car" in Figure 6). A critical stress test is to replicate the experiment on a base pretrained model (Qwen-2.5-7B without distillation, or Llama-3-8B) using the same DeepScaleR-Preview-Dataset for GRPO training. If multiplex thinking provides similar gains on base models, it demonstrates generality and suggests the method is a broadly applicable reasoning primitive. If gains are substantially smaller or disappear, it suggests that multiplexing depends on the model already having structured internal representations that can process off-manifold embedding mixtures — a fundamental limitation on the method's applicability. The experiment requires no methodological changes, just swapping the base model and retraining from scratch.

Combining multiplex thinking with outer-loop parallel reasoning (self-consistency, Best-of-N). The paper positions multiplex thinking as "complementary to existing parallel reasoning strategies" (Section 6) but never tests the combination. A straightforward follow-up would run self-consistency (majority voting) over N multiplex trajectories and compare to self-consistency over N discrete trajectories, sweeping N across powers of 2 (1, 2, 4, ..., 64). The question is whether the per-step exploration within each multiplex trajectory is partially redundant with the trajectory-level diversity from sampling multiple rollouts — i.e., do the two forms of exploration compound multiplicatively, additively, or sub-additively? The entropy analysis (Section 3.3) suggests that multiplexing expands the effective action space per step from $|V|$ to $|V|^K$ , which is a within-trajectory expansion. Self-consistency over N trajectories provides between-trajectory diversity. If these are independent sources of variance, the combination should outperform either alone. If they are partially overlapping (e.g., the main benefit of multiplexing is that it simulates having N parallel rollouts within a single sequence), then the marginal gain of adding outer-loop parallelism to multiplexing would be smaller than the marginal gain of adding it to discrete decoding. Testing this with Pass@k curves for multiplex + self-consistency vs. discrete + self-consistency would directly characterize the interaction and inform practical deployment decisions about how to allocate a fixed inference budget between per-step width and number of trajectories.

Entropy-preserving RL regularization schedules for multiplex training. The paper trains with zero KL penalty and zero entropy penalty, relying on the multiplex mechanism's built-in exploration to prevent policy collapse (Section 4, training hyperparameters). Table 4 shows this works — multiplex training has lower entropy reduction (5.82–7.09%) than discrete RL (9.44%). But the paper does not explore whether adding explicit entropy regularization would further improve results, or whether the current zero-penalty setting is already optimal. A systematic sweep over entropy penalty coefficients (0.0, 0.001, 0.01, 0.1) and KL penalty coefficients (same range) for both multiplex and discrete training would characterize the entropy-sensitivity of each method. The hypothesis: multiplex training benefits less from explicit entropy bonuses because the K-way sampling already provides structured exploration, so the optimal entropy coefficient for multiplex should be lower (or zero) compared to discrete. If this holds, it provides practical guidance — with multiplex tokens, you can safely remove entropy regularization, simplifying the training setup. If multiplex benefits from entropy bonuses at similar levels as discrete, it suggests the built-in exploration is insufficient on its own and the method should be combined with standard exploration-promoting techniques.

Wall-clock and memory profiling to ground the efficiency claims. The paper's efficiency argument rests on sequence length measurements (Figure 5, Table 3) and the claim that "increasing K does not require additional forward passes beyond sampling from the same logits distribution" (Section 5.3). A practical follow-up would measure actual GPU latency (milliseconds per generated token, both during the thinking phase and overall), peak GPU memory usage during training and inference, and total training wall-clock time for multiplex vs. discrete training at matched hyperparameters. The question: does the reduced sequence length translate to proportionally reduced wall-clock time, or does the per-step overhead of sampling K tokens, embedding them, averaging them, and computing K× the log-probability terms in the loss offset the savings? The paper runs on 8× NVIDIA DGX B200 GPUs (Appendix A.1), so profiling is straightforward. Key metrics to report: (1) tokens-per-second during inference for K=1 vs. K=3, (2) training step time for discrete RL vs. multiplex GRPO with equal batch size, (3) peak memory for both. If multiplex training is substantially slower per step due to the larger computation graph (K× log-probability terms in the policy gradient), the efficiency claim needs substantial qualification even if the sequence lengths are shorter.

Practical Applications and Downstream Use Cases

Drop-in inference-time improvement for existing reasoning models. The inference-only variant (Multiplex Thinking-I, Table 2) can be deployed on any compatible model without fine-tuning, providing roughly 0.5–5 point accuracy improvements over standard discrete CoT across math benchmarks while producing shorter output sequences (Table 3: 40.5% average accuracy at 4k tokens vs. 35.8% for Discrete CoT-4k). This is immediately useful for cost-sensitive API deployments where users pay per output token — shorter reasoning traces with equal or better accuracy directly reduce costs. The implementation requires no model modification, only a change to the decoding loop during the thinking phase. The K=3 default adds negligible per-step overhead (sampling three tokens from the same logits distribution and averaging embeddings), and the transition to discrete answer generation is handled automatically by the [eot] argmax rule. For teams using models like DeepSeek-R1-Distill-Qwen-7B in production, switching from discrete CoT to Multiplex Thinking-I is a low-risk, zero-training change that the paper's data suggests will improve both accuracy and token efficiency.

Cost-efficient data generation for self-improvement pipelines. Methods like STaR (Zelikman et al., 2022), ReST (Gulcehre et al., 2023), and DeepScaleR (Luo et al., 2025) generate reasoning traces from a model, filter for correctness, and fine-tune on the successful traces. The cost of this data generation scales with the number of sampled trajectories needed to find correct solutions. The Pass@k curves (Figure 2) show that multiplex thinking achieves a given Pass@k accuracy with substantially fewer trajectories than discrete CoT — on AIME 2025 (7B), reaching 50% Pass@k requires roughly 256 multiplex rollouts but over 1024 discrete rollouts (estimated from Figure 2, top row, second column). For a self-improvement pipeline generating 40,000 training traces, this translates to a ~4× reduction in generation cost at fixed correctness filtering rate. The multiplex trajectories are also shorter on average (Figure 5), further compounding the savings. The training framework already exists (the paper's GRPO setup), so a self-improvement loop would alternate between: (1) generating multiplex trajectories from the current model, (2) filtering for correctness using verifiable rewards, (3) fine-tuning (either discrete or multiplex) on correct trajectories, and (4) repeating.

On-device or low-latency reasoning with smaller models. The paper shows that multiplex thinking benefits scale with model size (gains are larger at 7B than at 1.5B), but the 1.5B model with multiplex thinking still outperforms the 1.5B discrete baseline across all benchmarks (Table 1, top block: e.g., 67.5 vs. 66.7 on MATH-500, 26.2 vs. 24.3 on Minerva). For edge deployment scenarios where a 1.5B model is the largest that fits in memory or meets latency requirements, switching to multiplex thinking provides a "free" accuracy improvement without increasing model size, sequence length, or per-token latency (since the forward pass per reasoning step is unchanged — only the embedding fed as input changes). The sequence length reduction (Figure 5) is particularly valuable in latency-sensitive settings: shorter responses mean the user waits less time for the final answer, even if per-step processing time is identical. A 1.5B model with multiplex thinking could serve as a local on-device reasoner for moderately difficult math problems, with hard problems escalated to a cloud-based larger model — and the multiplex model's higher Pass@k (Figure 2, top row for 1.5B) means fewer problems need escalation.

Evaluation of exploration quality in RL-trained reasoning models. The Pass@k metric (Figure 2) is already used to measure the exploration upper bound of reasoning models, but multiplex thinking introduces a new diagnostic: the gap between multiplex and discrete Pass@k curves at high k reveals how much of the model's exploration failure is due to the single-token bottleneck (abandoning high-entropy alternatives) versus genuine capability gaps (no correct solution in the distribution at all). If a model shows a large multiplex-discrete gap at k=1024 on a benchmark, it means the distribution contains correct solutions that discrete sampling struggles to find — the model "knows" the answer but can't surface it due to depth-first search limitations. If the gap is small, the model genuinely lacks the capability, and no amount of per-step exploration will help. This makes multiplex thinking a diagnostic tool for reasoning model evaluation: by comparing discrete and multiplex Pass@k curves, practitioners can distinguish between exploration bottlenecks and capability bottlenecks, guiding decisions about whether to invest in better search strategies or better pretraining. The paper's data already demonstrates this diagnostic: on AIME 2025 (7B), the ~15-point gap at k=1024 (55% vs. 40%) indicates an exploration bottleneck; on MATH-500 (7B), the small gap at high k indicates a capability ceiling (~90% for both methods, Figure 2 bottom row). This diagnostic use case requires no additional experiments beyond what the paper already reports — it is an interpretative framework that practitioners can apply to their own models and benchmarks.